2024-09-19T05:20:48.923 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T05:20:48.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:20:49.048 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911677' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBABHHSnugodJHOwMXN81m+caFk9wdDemd2rcwQr9BWyhO51fnm5mYuv3cshDHNCUUt3L2t7UMXAxXCAr3su91M0= smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEr9f4elCqu4N7hU9md2iaENmvuxgIAJdXmxYoc6q/Vi1IzJ+ZuZiiJ69cz8jwLo5UUgY7naSmrAUhPb32lpDFE= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCglE4fN2wWVMTB5Tz44FQpQOLFIV+Z/m5e46VdZzh0Wn/YDdwLs9bW2gOleDS2aZdxIouvYSRMNSYACrsBT6HQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T05:20:49.049 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T05:20:49.049 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T05:20:49.050 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T05:20:49.051 INFO:teuthology.task.internal:Checking packages... 2024-09-19T05:20:49.073 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T05:20:49.073 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T05:20:49.073 INFO:teuthology.packaging:ref: None 2024-09-19T05:20:49.074 INFO:teuthology.packaging:tag: None 2024-09-19T05:20:49.074 INFO:teuthology.packaging:branch: squid 2024-09-19T05:20:49.074 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:20:49.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T05:20:49.299 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T05:20:49.301 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T05:20:49.307 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T05:20:49.307 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T05:20:49.340 INFO:teuthology.task.internal:Saving configuration 2024-09-19T05:20:49.359 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T05:20:49.383 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T05:20:49.405 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:15:13.875691', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBABHHSnugodJHOwMXN81m+caFk9wdDemd2rcwQr9BWyhO51fnm5mYuv3cshDHNCUUt3L2t7UMXAxXCAr3su91M0='} 2024-09-19T05:20:49.425 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:15:13.874560', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEr9f4elCqu4N7hU9md2iaENmvuxgIAJdXmxYoc6q/Vi1IzJ+ZuZiiJ69cz8jwLo5UUgY7naSmrAUhPb32lpDFE='} 2024-09-19T05:20:49.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:15:13.876639', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCglE4fN2wWVMTB5Tz44FQpQOLFIV+Z/m5e46VdZzh0Wn/YDdwLs9bW2gOleDS2aZdxIouvYSRMNSYACrsBT6HQ='} 2024-09-19T05:20:49.443 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T05:20:49.449 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T05:20:49.449 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T05:20:49.450 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T05:20:49.450 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T05:20:49.574 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9dc93c7be0>, signals=[15]) 2024-09-19T05:20:49.574 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T05:20:49.580 INFO:teuthology.task.internal:Opening connections... 2024-09-19T05:20:49.580 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:20:49.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:20:49.662 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-19T05:20:49.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:20:49.741 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-09-19T05:20:49.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:20:49.820 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T05:20:49.826 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-09-19T05:20:49.845 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-09-19T05:20:49.845 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-09-19T05:20:49.902 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:20:49.903 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:20:49.904 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:20:49.904 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:20:49.904 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:20:49.904 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-09-19T05:20:49.926 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-19T05:20:49.945 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-19T05:20:49.945 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-19T05:20:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-09-19T05:20:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-09-19T05:20:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-09-19T05:20:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:20:50.003 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-09-19T05:20:50.004 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:20:50.004 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:20:50.005 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:20:50.005 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-19T05:20:50.026 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-09-19T05:20:50.045 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-09-19T05:20:50.045 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:20:50.103 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:20:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:20:50.104 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-09-19T05:20:50.126 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T05:20:50.138 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T05:20:50.165 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T05:20:50.165 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:20:50.168 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:20:50.170 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:20:50.187 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T05:20:50.198 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T05:20:50.198 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:20:50.226 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:20:50.228 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:20:50.366 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T05:20:50.446 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T05:20:50.461 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T05:20:50.462 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T05:20:50.462 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T05:20:50.462 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:20:50.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:20:50.480 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-19T05:20:50.538 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:20:50.538 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-503.el9.x86_64 2024-09-19T05:20:50.539 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-09-19T05:20:52.737 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-19T05:20:56.039 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 20 MB 00:02 2024-09-19T05:21:00.431 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-19T05:21:01.948 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-19T05:21:03.211 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-19T05:21:08.842 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T05:21:09.027 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Sep 2024 05:21:08 AM UTC. 2024-09-19T05:21:10.815 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:21:10.815 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:21:10.908 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-19T05:21:10.915 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-19T05:21:10.916 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2024-09-19T05:21:10.917 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:21:10.917 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 79 M 2024-09-19T05:21:10.917 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 126 M 2024-09-19T05:21:10.917 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-19T05:21:11.052 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 401 kB/s | 36 kB 00:00 2024-09-19T05:21:11.386 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-19T05:21:11.786 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 42 MB/s | 35 MB 00:00 2024-09-19T05:21:12.236 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-19T05:21:12.236 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:21:12.236 INFO:teuthology.orchestra.run.smithi038.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-19T05:21:12.945 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-19T05:21:13.367 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-19T05:21:13.367 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-19T05:21:14.334 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-19T05:21:14.335 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-19T05:21:16.873 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-19T05:21:17.558 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:21:17.648 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:21:20.400 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:21:20.525 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:21:25.652 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:21:25.684 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:21:30.906 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:22:10.182 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:22:10.223 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:22:10.580 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:22:10.580 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:22:10.581 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:22:10.581 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:22:11.435 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-19T05:22:11.811 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:22:11.901 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.901 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:22:11.901 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T05:22:11.902 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T05:22:11.902 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T05:22:11.902 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T05:22:11.902 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T05:22:11.902 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:22:11.961 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:22:11.961 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:22:11.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 82.el9 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8274670 2024-09-19T05:22:11.994 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T05:22:11.995 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-09-19T05:22:11.996 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:22:11.996 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T05:22:11.997 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T05:22:11.997 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 5.14.0-508.el9.x86_64 2024-09-19T05:22:11.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T05:22:12.629 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-09-19T05:22:13.978 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T05:22:14.007 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-09-19T05:22:14.008 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-09-19T05:22:14.041 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.hqyQJ7yZMX 2024-09-19T05:22:14.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hqyQJ7yZMX 2024-09-19T05:22:14.111 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.hqyQJ7yZMX 2024-09-19T05:22:14.248 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.hqyQJ7yZMX is 6KB 2024-09-19T05:22:14.302 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.hqyQJ7yZMX 2024-09-19T05:22:14.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T05:22:14.390 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T05:22:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T05:22:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T05:22:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T05:22:14.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T05:22:14.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-09-19T05:22:44.531 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T05:22:44.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:22:44.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:23:03.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-09-19T05:23:12.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:23:12.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:24:12.106 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T05:24:24.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:24:24.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:24:24.511 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-19T05:24:24.896 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:24:24.896 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T05:24:24.896 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-19T05:24:24.913 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-508.el9.x86_64 2024-09-19T05:24:24.913 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T05:24:24.914 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T05:24:24.914 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T05:24:25.915 DEBUG:teuthology.parallel:result is None 2024-09-19T05:24:25.915 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T05:24:25.923 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T05:24:25.924 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:24:25.926 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:24:25.929 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:24:25.947 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T05:24:25.966 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T05:24:26.017 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T05:24:26.017 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:24:26.020 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:24:26.022 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:24:26.060 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T05:24:26.066 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T05:24:26.067 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:24:26.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:24:26.110 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:24:26.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:24:26.127 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:24:26.130 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:24:26.152 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:24:26.188 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.209 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.215 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.222 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.232 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:24:26.234 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T05:24:26.240 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T05:24:26.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:24:26.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:24:26.253 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:24:26.294 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T05:24:26.303 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T05:24:26.303 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:24:26.314 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:24:26.322 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:24:26.351 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:26.414 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:26.464 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:26.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:26.591 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:24:26.591 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:24:26.658 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:26.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:26.760 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:26.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:26.887 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:24:26.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:24:26.954 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:26.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:24:27.057 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:27.140 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:24:27.184 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:24:27.184 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:24:27.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-09-19T05:24:27.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-19T05:24:27.255 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-09-19T05:24:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:24:27.302 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:24:27.326 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:24:27.775 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T05:24:27.782 INFO:teuthology.task.internal:Starting timer... 2024-09-19T05:24:27.783 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T05:24:27.853 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T05:24:27.902 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T05:24:27.902 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-09-19T05:24:27.965 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-09-19T05:24:27.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-09-19T05:24:28.026 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-09-19T05:24:28.029 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-09-19T05:24:28.099 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-09-19T05:24:28.101 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T05:24:28.101 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-09-19T05:24:28.131 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-09-19T05:24:28.132 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-09-19T05:24:28.160 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-09-19T05:24:28.160 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-09-19T05:24:28.192 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-09-19T05:24:28.192 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-09-19T05:24:28.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:24:28.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:24:28.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:24:28.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:24:28.253 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:24:28.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:24:28.283 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T05:24:28.283 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:24:28.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:24:28.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:24:28.373 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T05:24:28.383 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T05:24:28.397 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T05:24:28.400 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi047.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T05:28:39.604 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-09-19T05:28:39.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:28:39.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:28:39.691 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-19T05:28:39.775 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-19T05:28:39.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-19T05:28:39.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:28:39.858 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-19T05:28:39.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-19T05:28:39.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-09-19T05:28:39.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:28:40.020 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-09-19T05:28:40.100 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-09-19T05:28:40.100 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T05:28:40.111 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T05:28:40.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:28:40.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:28:40.115 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:28:40.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:28:40.117 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:28:40.117 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:28:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:28:40.148 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:28:40.164 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:28:40.164 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:28:40.169 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:28:40.184 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:28:40.190 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-09-19T05:28:40.192 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-09-19T05:28:40.202 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-09-19T05:28:40.205 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-09-19T05:28:40.208 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-09-19T05:28:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:28:40.219 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-09-19T05:28:40.221 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:28:40.231 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:28:40.234 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:28:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:28:40.247 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:28:40.286 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.289 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.302 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:28:40.304 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.306 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.307 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:28:40.307 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-19T05:28:40.307 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.307 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.307 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.308 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:28:40.308 INFO:teuthology.run_tasks:Running task install... 2024-09-19T05:28:40.316 DEBUG:teuthology.task.install:project ceph 2024-09-19T05:28:40.317 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T05:28:40.317 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T05:28:40.317 INFO:teuthology.task.install:Using flavor: default 2024-09-19T05:28:40.322 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T05:28:40.323 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T05:28:40.323 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:28:40.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:28:40.326 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:28:40.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:28:40.328 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:28:40.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:28:40.474 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:28:40.474 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:28:40.485 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:28:40.485 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:28:40.489 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:28:40.489 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:28:40.585 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:28:40.585 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:28:40.585 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:28:40.597 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:28:40.597 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:28:40.597 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:28:40.601 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:28:40.602 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:28:40.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:28:40.613 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:28:40.613 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:28:40.622 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:28:40.623 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:28:40.628 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:28:40.628 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:28:40.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:28:40.688 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:28:40.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:28:40.760 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:28:40.763 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:28:40.768 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:28:40.825 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-09-19T05:28:40.827 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-09-19T05:28:40.827 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-09-19T05:28:40.829 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-09-19T05:28:40.832 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-09-19T05:28:40.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-09-19T05:28:41.056 INFO:teuthology.orchestra.run.smithi179.stdout:45 files removed 2024-09-19T05:28:41.063 INFO:teuthology.orchestra.run.smithi038.stdout:45 files removed 2024-09-19T05:28:41.064 INFO:teuthology.orchestra.run.smithi047.stdout:45 files removed 2024-09-19T05:28:41.083 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:28:41.091 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:28:41.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:28:41.628 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-19T05:28:41.652 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T05:28:41.672 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-19T05:28:41.912 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T05:28:41.945 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T05:28:41.956 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-19T05:28:42.187 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T05:28:42.214 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-19T05:28:42.363 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-09-19T05:28:42.646 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-19T05:28:42.683 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-19T05:28:42.962 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T05:28:44.658 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-19T05:28:47.488 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 20 MB 00:03 2024-09-19T05:28:48.156 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-19T05:28:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-19T05:28:51.549 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-19T05:28:52.545 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T05:28:52.690 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T05:28:53.664 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-19T05:28:54.466 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-19T05:28:55.114 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 19 kB 00:00 2024-09-19T05:28:55.462 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T05:28:56.102 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T05:28:58.059 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T05:29:00.205 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:00.205 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:00.219 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:29:00.220 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:29:00.221 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:29:00.222 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:29:00.224 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:29:00.226 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:29:00.227 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:29:00.290 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-09-19T05:29:00.300 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-09-19T05:29:00.300 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-09-19T05:29:00.300 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:29:00.301 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:29:00.302 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:29:00.303 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:29:00.304 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:29:00.305 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:29:00.306 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:29:00.307 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:29:00.308 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:29:00.309 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:29:00.310 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:29:00.311 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:29:00.312 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:29:00.313 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout:Install 148 Packages 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 8 Packages 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:00.314 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 226 M 2024-09-19T05:29:00.315 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-09-19T05:29:00.935 INFO:teuthology.orchestra.run.smithi179.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 44 kB/s | 6.6 kB 00:00 2024-09-19T05:29:01.136 INFO:teuthology.orchestra.run.smithi179.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.4 MB/s | 896 kB 00:00 2024-09-19T05:29:01.186 INFO:teuthology.orchestra.run.smithi179.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.8 MB/s | 145 kB 00:00 2024-09-19T05:29:01.193 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T05:29:01.420 INFO:teuthology.orchestra.run.smithi179.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 8.6 MB/s | 5.4 MB 00:00 2024-09-19T05:29:01.470 INFO:teuthology.orchestra.run.smithi179.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 8.4 MB/s | 2.4 MB 00:00 2024-09-19T05:29:01.562 INFO:teuthology.orchestra.run.smithi179.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-19T05:29:01.783 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T05:29:01.903 INFO:teuthology.orchestra.run.smithi179.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-19T05:29:02.295 INFO:teuthology.orchestra.run.smithi179.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 14 MB/s | 21 MB 00:01 2024-09-19T05:29:02.337 INFO:teuthology.orchestra.run.smithi179.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 608 kB/s | 25 kB 00:00 2024-09-19T05:29:02.588 INFO:teuthology.orchestra.run.smithi179.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 16 MB/s | 17 MB 00:01 2024-09-19T05:29:02.638 INFO:teuthology.orchestra.run.smithi179.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T05:29:02.755 INFO:teuthology.orchestra.run.smithi179.stdout:(12/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 12 MB/s | 11 MB 00:00 2024-09-19T05:29:02.780 INFO:teuthology.orchestra.run.smithi179.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 190 kB/s | 27 kB 00:00 2024-09-19T05:29:02.797 INFO:teuthology.orchestra.run.smithi179.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 760 kB/s | 32 kB 00:00 2024-09-19T05:29:02.839 INFO:teuthology.orchestra.run.smithi179.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T05:29:02.864 INFO:teuthology.orchestra.run.smithi179.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T05:29:02.907 INFO:teuthology.orchestra.run.smithi179.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.9 MB/s | 127 kB 00:00 2024-09-19T05:29:02.932 INFO:teuthology.orchestra.run.smithi179.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T05:29:02.982 INFO:teuthology.orchestra.run.smithi179.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 902 kB/s | 45 kB 00:00 2024-09-19T05:29:03.074 INFO:teuthology.orchestra.run.smithi179.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T05:29:03.174 INFO:teuthology.orchestra.run.smithi179.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-19T05:29:03.200 INFO:teuthology.orchestra.run.smithi179.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.3 MB/s | 161 kB 00:00 2024-09-19T05:29:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T05:29:03.250 INFO:teuthology.orchestra.run.smithi179.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T05:29:03.275 INFO:teuthology.orchestra.run.smithi179.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T05:29:03.310 INFO:teuthology.orchestra.run.smithi179.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T05:29:03.377 INFO:teuthology.orchestra.run.smithi179.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T05:29:03.436 INFO:teuthology.orchestra.run.smithi179.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-19T05:29:03.461 INFO:teuthology.orchestra.run.smithi179.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 381 kB/s | 32 kB 00:00 2024-09-19T05:29:03.471 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:03.472 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:03.484 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:29:03.485 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:29:03.486 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:29:03.486 INFO:teuthology.orchestra.run.smithi179.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T05:29:03.487 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:29:03.488 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:29:03.491 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:29:03.492 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:29:03.559 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-19T05:29:03.572 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T05:29:03.572 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-09-19T05:29:03.572 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:29:03.573 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:29:03.574 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:29:03.575 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:29:03.576 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:29:03.577 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:29:03.578 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:29:03.579 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:29:03.580 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:29:03.581 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:29:03.582 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:29:03.583 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:29:03.584 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 8 Packages 2024-09-19T05:29:03.585 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 226 M 2024-09-19T05:29:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-19T05:29:03.787 INFO:teuthology.orchestra.run.smithi179.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 11 MB/s | 3.6 MB 00:00 2024-09-19T05:29:03.912 INFO:teuthology.orchestra.run.smithi179.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-19T05:29:03.937 INFO:teuthology.orchestra.run.smithi179.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 1.6 MB/s | 249 kB 00:00 2024-09-19T05:29:03.963 INFO:teuthology.orchestra.run.smithi179.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 987 kB/s | 49 kB 00:00 2024-09-19T05:29:03.988 INFO:teuthology.orchestra.run.smithi179.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 327 kB/s | 16 kB 00:00 2024-09-19T05:29:04.013 INFO:teuthology.orchestra.run.smithi179.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T05:29:04.021 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:04.023 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:29:04.037 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:29:04.038 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:29:04.039 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:29:04.040 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:29:04.042 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:29:04.045 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:29:04.046 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:29:04.055 INFO:teuthology.orchestra.run.smithi179.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 438 kB/s | 18 kB 00:00 2024-09-19T05:29:04.097 INFO:teuthology.orchestra.run.smithi179.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-19T05:29:04.113 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:29:04.133 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:29:04.134 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:29:04.135 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:29:04.136 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:29:04.137 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:29:04.138 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:29:04.139 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:29:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:29:04.141 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:29:04.142 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:29:04.143 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:29:04.144 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:29:04.145 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 8 Packages 2024-09-19T05:29:04.146 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:04.147 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 226 M 2024-09-19T05:29:04.147 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-19T05:29:04.206 INFO:teuthology.orchestra.run.smithi179.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-19T05:29:04.227 INFO:teuthology.orchestra.run.smithi038.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-19T05:29:04.273 INFO:teuthology.orchestra.run.smithi179.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-19T05:29:04.299 INFO:teuthology.orchestra.run.smithi179.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-19T05:29:04.365 INFO:teuthology.orchestra.run.smithi179.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-19T05:29:04.416 INFO:teuthology.orchestra.run.smithi179.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-19T05:29:04.441 INFO:teuthology.orchestra.run.smithi179.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T05:29:04.453 INFO:teuthology.orchestra.run.smithi038.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T05:29:04.503 INFO:teuthology.orchestra.run.smithi038.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.8 MB/s | 145 kB 00:00 2024-09-19T05:29:04.543 INFO:teuthology.orchestra.run.smithi047.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T05:29:04.745 INFO:teuthology.orchestra.run.smithi038.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 9.9 MB/s | 2.4 MB 00:00 2024-09-19T05:29:04.751 INFO:teuthology.orchestra.run.smithi047.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.2 MB/s | 896 kB 00:00 2024-09-19T05:29:04.810 INFO:teuthology.orchestra.run.smithi047.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T05:29:04.862 INFO:teuthology.orchestra.run.smithi038.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-09-19T05:29:04.866 INFO:teuthology.orchestra.run.smithi179.stdout:(45/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T05:29:04.892 INFO:teuthology.orchestra.run.smithi179.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 277 kB/s | 125 kB 00:00 2024-09-19T05:29:04.925 INFO:teuthology.orchestra.run.smithi179.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 265 kB/s | 135 kB 00:00 2024-09-19T05:29:04.945 INFO:teuthology.orchestra.run.smithi038.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-19T05:29:05.009 INFO:teuthology.orchestra.run.smithi179.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-19T05:29:05.117 INFO:teuthology.orchestra.run.smithi179.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-19T05:29:05.119 INFO:teuthology.orchestra.run.smithi047.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-19T05:29:05.153 INFO:teuthology.orchestra.run.smithi179.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-09-19T05:29:05.176 INFO:teuthology.orchestra.run.smithi179.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-19T05:29:05.194 INFO:teuthology.orchestra.run.smithi047.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 6.9 MB/s | 5.4 MB 00:00 2024-09-19T05:29:05.202 INFO:teuthology.orchestra.run.smithi179.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-19T05:29:05.227 INFO:teuthology.orchestra.run.smithi179.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-19T05:29:05.260 INFO:teuthology.orchestra.run.smithi179.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-19T05:29:05.286 INFO:teuthology.orchestra.run.smithi179.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-19T05:29:05.287 INFO:teuthology.orchestra.run.smithi038.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-19T05:29:05.311 INFO:teuthology.orchestra.run.smithi179.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-19T05:29:05.345 INFO:teuthology.orchestra.run.smithi179.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T05:29:05.353 INFO:teuthology.orchestra.run.smithi047.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-19T05:29:05.370 INFO:teuthology.orchestra.run.smithi179.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-19T05:29:05.404 INFO:teuthology.orchestra.run.smithi179.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-19T05:29:05.437 INFO:teuthology.orchestra.run.smithi179.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-19T05:29:05.464 INFO:teuthology.orchestra.run.smithi179.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T05:29:05.488 INFO:teuthology.orchestra.run.smithi179.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T05:29:05.513 INFO:teuthology.orchestra.run.smithi179.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T05:29:05.539 INFO:teuthology.orchestra.run.smithi179.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T05:29:05.563 INFO:teuthology.orchestra.run.smithi179.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-19T05:29:05.614 INFO:teuthology.orchestra.run.smithi179.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-09-19T05:29:05.706 INFO:teuthology.orchestra.run.smithi179.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-19T05:29:05.748 INFO:teuthology.orchestra.run.smithi179.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T05:29:05.815 INFO:teuthology.orchestra.run.smithi179.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-19T05:29:05.840 INFO:teuthology.orchestra.run.smithi179.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-19T05:29:05.865 INFO:teuthology.orchestra.run.smithi179.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 5.6 MB/s | 910 kB 00:00 2024-09-19T05:29:05.908 INFO:teuthology.orchestra.run.smithi179.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-19T05:29:05.933 INFO:teuthology.orchestra.run.smithi179.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T05:29:05.944 INFO:teuthology.orchestra.run.smithi047.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 6.3 MB/s | 4.7 MB 00:00 2024-09-19T05:29:05.966 INFO:teuthology.orchestra.run.smithi179.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T05:29:06.042 INFO:teuthology.orchestra.run.smithi179.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-19T05:29:06.067 INFO:teuthology.orchestra.run.smithi179.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T05:29:06.092 INFO:teuthology.orchestra.run.smithi179.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T05:29:06.117 INFO:teuthology.orchestra.run.smithi179.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-19T05:29:06.143 INFO:teuthology.orchestra.run.smithi179.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-19T05:29:06.168 INFO:teuthology.orchestra.run.smithi179.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T05:29:06.193 INFO:teuthology.orchestra.run.smithi179.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-19T05:29:06.229 INFO:teuthology.orchestra.run.smithi038.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 10 MB/s | 21 MB 00:02 2024-09-19T05:29:06.268 INFO:teuthology.orchestra.run.smithi179.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T05:29:06.294 INFO:teuthology.orchestra.run.smithi179.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T05:29:06.313 INFO:teuthology.orchestra.run.smithi038.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 302 kB/s | 25 kB 00:00 2024-09-19T05:29:06.319 INFO:teuthology.orchestra.run.smithi179.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T05:29:06.344 INFO:teuthology.orchestra.run.smithi179.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T05:29:06.370 INFO:teuthology.orchestra.run.smithi179.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-19T05:29:06.403 INFO:teuthology.orchestra.run.smithi179.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T05:29:06.429 INFO:teuthology.orchestra.run.smithi179.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-19T05:29:06.429 INFO:teuthology.orchestra.run.smithi038.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 9.3 MB/s | 11 MB 00:01 2024-09-19T05:29:06.462 INFO:teuthology.orchestra.run.smithi179.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T05:29:06.487 INFO:teuthology.orchestra.run.smithi179.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 875 kB/s | 51 kB 00:00 2024-09-19T05:29:06.504 INFO:teuthology.orchestra.run.smithi179.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-19T05:29:06.530 INFO:teuthology.orchestra.run.smithi038.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.2 MB/s | 228 kB 00:00 2024-09-19T05:29:06.580 INFO:teuthology.orchestra.run.smithi179.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T05:29:06.605 INFO:teuthology.orchestra.run.smithi179.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-19T05:29:06.630 INFO:teuthology.orchestra.run.smithi179.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-19T05:29:06.664 INFO:teuthology.orchestra.run.smithi179.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T05:29:06.689 INFO:teuthology.orchestra.run.smithi179.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-09-19T05:29:06.715 INFO:teuthology.orchestra.run.smithi179.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-09-19T05:29:06.738 INFO:teuthology.orchestra.run.smithi038.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 9.4 MB/s | 17 MB 00:01 2024-09-19T05:29:06.748 INFO:teuthology.orchestra.run.smithi179.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T05:29:06.764 INFO:teuthology.orchestra.run.smithi038.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 116 kB/s | 27 kB 00:00 2024-09-19T05:29:06.773 INFO:teuthology.orchestra.run.smithi179.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-09-19T05:29:06.789 INFO:teuthology.orchestra.run.smithi038.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 634 kB/s | 32 kB 00:00 2024-09-19T05:29:06.799 INFO:teuthology.orchestra.run.smithi179.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T05:29:06.824 INFO:teuthology.orchestra.run.smithi179.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T05:29:06.847 INFO:teuthology.orchestra.run.smithi038.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-19T05:29:06.858 INFO:teuthology.orchestra.run.smithi179.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-19T05:29:06.878 INFO:teuthology.orchestra.run.smithi047.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 8.6 MB/s | 21 MB 00:02 2024-09-19T05:29:06.881 INFO:teuthology.orchestra.run.smithi038.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.7 MB/s | 163 kB 00:00 2024-09-19T05:29:06.906 INFO:teuthology.orchestra.run.smithi038.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.1 MB/s | 127 kB 00:00 2024-09-19T05:29:06.949 INFO:teuthology.orchestra.run.smithi038.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 7.4 MB/s | 506 kB 00:00 2024-09-19T05:29:07.015 INFO:teuthology.orchestra.run.smithi038.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 680 kB/s | 45 kB 00:00 2024-09-19T05:29:07.053 INFO:teuthology.orchestra.run.smithi047.stdout:(9/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 9.9 MB/s | 17 MB 00:01 2024-09-19T05:29:07.078 INFO:teuthology.orchestra.run.smithi047.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 126 kB/s | 25 kB 00:00 2024-09-19T05:29:07.082 INFO:teuthology.orchestra.run.smithi038.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T05:29:07.083 INFO:teuthology.orchestra.run.smithi179.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T05:29:07.133 INFO:teuthology.orchestra.run.smithi179.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T05:29:07.166 INFO:teuthology.orchestra.run.smithi038.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.9 MB/s | 161 kB 00:00 2024-09-19T05:29:07.175 INFO:teuthology.orchestra.run.smithi179.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-19T05:29:07.200 INFO:teuthology.orchestra.run.smithi179.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 790 kB/s | 19 kB 00:00 2024-09-19T05:29:07.204 INFO:teuthology.orchestra.run.smithi047.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 8.4 MB/s | 11 MB 00:01 2024-09-19T05:29:07.229 INFO:teuthology.orchestra.run.smithi047.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.5 MB/s | 228 kB 00:00 2024-09-19T05:29:07.250 INFO:teuthology.orchestra.run.smithi038.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 3.8 MB/s | 320 kB 00:00 2024-09-19T05:29:07.287 INFO:teuthology.orchestra.run.smithi047.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 324 kB/s | 27 kB 00:00 2024-09-19T05:29:07.313 INFO:teuthology.orchestra.run.smithi047.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 380 kB/s | 32 kB 00:00 2024-09-19T05:29:07.359 INFO:teuthology.orchestra.run.smithi179.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-19T05:29:07.383 INFO:teuthology.orchestra.run.smithi038.stdout:(23/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-19T05:29:07.384 INFO:teuthology.orchestra.run.smithi179.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-19T05:29:07.396 INFO:teuthology.orchestra.run.smithi047.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.9 MB/s | 163 kB 00:00 2024-09-19T05:29:07.408 INFO:teuthology.orchestra.run.smithi038.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 1.8 MB/s | 300 kB 00:00 2024-09-19T05:29:07.410 INFO:teuthology.orchestra.run.smithi179.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T05:29:07.422 INFO:teuthology.orchestra.run.smithi047.stdout:(16/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-19T05:29:07.435 INFO:teuthology.orchestra.run.smithi179.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T05:29:07.460 INFO:teuthology.orchestra.run.smithi179.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-19T05:29:07.461 INFO:teuthology.orchestra.run.smithi038.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T05:29:07.472 INFO:teuthology.orchestra.run.smithi047.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.6 MB/s | 127 kB 00:00 2024-09-19T05:29:07.492 INFO:teuthology.orchestra.run.smithi038.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-19T05:29:07.502 INFO:teuthology.orchestra.run.smithi179.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-19T05:29:07.514 INFO:teuthology.orchestra.run.smithi047.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T05:29:07.544 INFO:teuthology.orchestra.run.smithi179.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T05:29:07.569 INFO:teuthology.orchestra.run.smithi179.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T05:29:07.593 INFO:teuthology.orchestra.run.smithi038.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-19T05:29:07.595 INFO:teuthology.orchestra.run.smithi179.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T05:29:07.614 INFO:teuthology.orchestra.run.smithi047.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 451 kB/s | 45 kB 00:00 2024-09-19T05:29:07.620 INFO:teuthology.orchestra.run.smithi179.stdout:(116/156): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-09-19T05:29:07.645 INFO:teuthology.orchestra.run.smithi179.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 287 kB/s | 14 kB 00:00 2024-09-19T05:29:07.687 INFO:teuthology.orchestra.run.smithi179.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-19T05:29:07.715 INFO:teuthology.orchestra.run.smithi047.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T05:29:07.726 INFO:teuthology.orchestra.run.smithi038.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 238 kB/s | 32 kB 00:00 2024-09-19T05:29:07.729 INFO:teuthology.orchestra.run.smithi179.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.4 MB/s | 303 kB 00:00 2024-09-19T05:29:07.755 INFO:teuthology.orchestra.run.smithi179.stdout:(120/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-09-19T05:29:07.780 INFO:teuthology.orchestra.run.smithi179.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T05:29:07.805 INFO:teuthology.orchestra.run.smithi179.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T05:29:07.815 INFO:teuthology.orchestra.run.smithi047.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T05:29:07.818 INFO:teuthology.orchestra.run.smithi038.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.6 MB/s | 146 kB 00:00 2024-09-19T05:29:07.830 INFO:teuthology.orchestra.run.smithi179.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-19T05:29:07.855 INFO:teuthology.orchestra.run.smithi179.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 716 kB/s | 18 kB 00:00 2024-09-19T05:29:07.881 INFO:teuthology.orchestra.run.smithi179.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 920 kB/s | 23 kB 00:00 2024-09-19T05:29:07.894 INFO:teuthology.orchestra.run.smithi038.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 7.2 MB/s | 3.1 MB 00:00 2024-09-19T05:29:07.906 INFO:teuthology.orchestra.run.smithi179.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-19T05:29:07.915 INFO:teuthology.orchestra.run.smithi047.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 3.1 MB/s | 320 kB 00:00 2024-09-19T05:29:07.931 INFO:teuthology.orchestra.run.smithi179.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-09-19T05:29:07.990 INFO:teuthology.orchestra.run.smithi179.stdout:(128/156): protobuf-compiler-3.14.0-13.el9.x86_ 762 kB/s | 863 kB 00:01 2024-09-19T05:29:08.049 INFO:teuthology.orchestra.run.smithi179.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-09-19T05:29:08.074 INFO:teuthology.orchestra.run.smithi179.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.3 MB/s | 79 kB 00:00 2024-09-19T05:29:08.091 INFO:teuthology.orchestra.run.smithi179.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-09-19T05:29:08.108 INFO:teuthology.orchestra.run.smithi179.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-19T05:29:08.142 INFO:teuthology.orchestra.run.smithi179.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-19T05:29:08.158 INFO:teuthology.orchestra.run.smithi179.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-19T05:29:08.175 INFO:teuthology.orchestra.run.smithi179.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-19T05:29:08.201 INFO:teuthology.orchestra.run.smithi179.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T05:29:08.224 INFO:teuthology.orchestra.run.smithi047.stdout:(23/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 7.1 MB/s | 5.3 MB 00:00 2024-09-19T05:29:08.226 INFO:teuthology.orchestra.run.smithi179.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-19T05:29:08.260 INFO:teuthology.orchestra.run.smithi179.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-19T05:29:08.285 INFO:teuthology.orchestra.run.smithi179.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T05:29:08.307 INFO:teuthology.orchestra.run.smithi047.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.2 MB/s | 99 kB 00:00 2024-09-19T05:29:08.310 INFO:teuthology.orchestra.run.smithi179.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-19T05:29:08.335 INFO:teuthology.orchestra.run.smithi179.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T05:29:08.387 INFO:teuthology.orchestra.run.smithi179.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 737 kB/s | 74 kB 00:00 2024-09-19T05:29:08.408 INFO:teuthology.orchestra.run.smithi047.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 851 kB/s | 85 kB 00:00 2024-09-19T05:29:08.411 INFO:teuthology.orchestra.run.smithi179.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-19T05:29:08.436 INFO:teuthology.orchestra.run.smithi179.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-19T05:29:08.462 INFO:teuthology.orchestra.run.smithi179.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-09-19T05:29:08.487 INFO:teuthology.orchestra.run.smithi179.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T05:29:08.511 INFO:teuthology.orchestra.run.smithi038.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 5.2 MB/s | 3.6 MB 00:00 2024-09-19T05:29:08.512 INFO:teuthology.orchestra.run.smithi179.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T05:29:08.571 INFO:teuthology.orchestra.run.smithi179.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T05:29:08.670 INFO:teuthology.orchestra.run.smithi038.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 9.5 MB/s | 7.4 MB 00:00 2024-09-19T05:29:08.694 INFO:teuthology.orchestra.run.smithi038.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 1.3 MB/s | 249 kB 00:00 2024-09-19T05:29:08.738 INFO:teuthology.orchestra.run.smithi179.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-09-19T05:29:08.745 INFO:teuthology.orchestra.run.smithi038.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 657 kB/s | 49 kB 00:00 2024-09-19T05:29:08.770 INFO:teuthology.orchestra.run.smithi038.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 218 kB/s | 16 kB 00:00 2024-09-19T05:29:08.837 INFO:teuthology.orchestra.run.smithi038.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.0 MB/s | 285 kB 00:00 2024-09-19T05:29:08.846 INFO:teuthology.orchestra.run.smithi179.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-09-19T05:29:08.922 INFO:teuthology.orchestra.run.smithi179.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 546 kB/s | 41 kB 00:00 2024-09-19T05:29:08.925 INFO:teuthology.orchestra.run.smithi047.stdout:(26/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.0 MB/s | 3.1 MB 00:00 2024-09-19T05:29:08.929 INFO:teuthology.orchestra.run.smithi038.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 199 kB/s | 18 kB 00:00 2024-09-19T05:29:08.962 INFO:teuthology.orchestra.run.smithi038.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-19T05:29:09.005 INFO:teuthology.orchestra.run.smithi179.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T05:29:09.016 INFO:teuthology.orchestra.run.smithi047.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-19T05:29:09.041 INFO:teuthology.orchestra.run.smithi047.stdout:(28/156): python3-rbd-19.1.1-361.g2f599a46.el9. 267 kB/s | 300 kB 00:01 2024-09-19T05:29:09.081 INFO:teuthology.orchestra.run.smithi179.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-19T05:29:09.096 INFO:teuthology.orchestra.run.smithi038.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-19T05:29:09.100 INFO:teuthology.orchestra.run.smithi047.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 380 kB/s | 32 kB 00:00 2024-09-19T05:29:09.198 INFO:teuthology.orchestra.run.smithi179.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-19T05:29:09.201 INFO:teuthology.orchestra.run.smithi038.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 191 kB 00:00 2024-09-19T05:29:09.238 INFO:teuthology.orchestra.run.smithi038.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-19T05:29:09.248 INFO:teuthology.orchestra.run.smithi179.stdout:(155/156): librbd1-19.1.1-361.g2f599a46.el9.x86 4.3 MB/s | 3.2 MB 00:00 2024-09-19T05:29:09.263 INFO:teuthology.orchestra.run.smithi038.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 253 kB 00:00 2024-09-19T05:29:09.301 INFO:teuthology.orchestra.run.smithi047.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 565 kB/s | 146 kB 00:00 2024-09-19T05:29:09.306 INFO:teuthology.orchestra.run.smithi179.stdout:(156/156): librados2-19.1.1-361.g2f599a46.el9.x 4.2 MB/s | 3.4 MB 00:00 2024-09-19T05:29:09.313 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:29:09.313 INFO:teuthology.orchestra.run.smithi179.stdout:Total 25 MB/s | 226 MB 00:09 2024-09-19T05:29:09.314 INFO:teuthology.orchestra.run.smithi038.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T05:29:09.356 INFO:teuthology.orchestra.run.smithi038.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-19T05:29:09.381 INFO:teuthology.orchestra.run.smithi038.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-19T05:29:09.398 INFO:teuthology.orchestra.run.smithi038.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-19T05:29:09.440 INFO:teuthology.orchestra.run.smithi038.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T05:29:09.617 INFO:teuthology.orchestra.run.smithi047.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.9 MB/s | 3.6 MB 00:00 2024-09-19T05:29:09.624 INFO:teuthology.orchestra.run.smithi038.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-19T05:29:09.666 INFO:teuthology.orchestra.run.smithi038.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-19T05:29:09.691 INFO:teuthology.orchestra.run.smithi038.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-19T05:29:09.716 INFO:teuthology.orchestra.run.smithi038.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-19T05:29:09.718 INFO:teuthology.orchestra.run.smithi047.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.4 MB/s | 249 kB 00:00 2024-09-19T05:29:09.775 INFO:teuthology.orchestra.run.smithi038.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-19T05:29:09.800 INFO:teuthology.orchestra.run.smithi038.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-19T05:29:09.801 INFO:teuthology.orchestra.run.smithi047.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 592 kB/s | 49 kB 00:00 2024-09-19T05:29:09.842 INFO:teuthology.orchestra.run.smithi038.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-19T05:29:09.884 INFO:teuthology.orchestra.run.smithi038.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-19T05:29:09.885 INFO:teuthology.orchestra.run.smithi047.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 197 kB/s | 16 kB 00:00 2024-09-19T05:29:09.951 INFO:teuthology.orchestra.run.smithi038.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-19T05:29:09.985 INFO:teuthology.orchestra.run.smithi047.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.8 MB/s | 285 kB 00:00 2024-09-19T05:29:09.993 INFO:teuthology.orchestra.run.smithi038.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T05:29:10.152 INFO:teuthology.orchestra.run.smithi047.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-19T05:29:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-19T05:29:10.227 INFO:teuthology.orchestra.run.smithi047.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 243 kB/s | 18 kB 00:00 2024-09-19T05:29:10.369 INFO:teuthology.orchestra.run.smithi038.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-19T05:29:10.410 INFO:teuthology.orchestra.run.smithi038.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 592 kB/s | 24 kB 00:00 2024-09-19T05:29:10.545 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-09-19T05:29:10.653 INFO:teuthology.orchestra.run.smithi047.stdout:(38/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 14 MB/s | 49 MB 00:03 2024-09-19T05:29:10.697 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-09-19T05:29:10.697 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-09-19T05:29:10.770 INFO:teuthology.orchestra.run.smithi047.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 147 kB/s | 80 kB 00:00 2024-09-19T05:29:10.853 INFO:teuthology.orchestra.run.smithi047.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T05:29:10.861 INFO:teuthology.orchestra.run.smithi038.stdout:(61/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 11 MB/s | 49 MB 00:04 2024-09-19T05:29:10.886 INFO:teuthology.orchestra.run.smithi038.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 100 kB/s | 47 kB 00:00 2024-09-19T05:29:10.936 INFO:teuthology.orchestra.run.smithi038.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-19T05:29:10.945 INFO:teuthology.orchestra.run.smithi047.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-19T05:29:10.962 INFO:teuthology.orchestra.run.smithi038.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-19T05:29:10.970 INFO:teuthology.orchestra.run.smithi047.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-19T05:29:10.987 INFO:teuthology.orchestra.run.smithi038.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-19T05:29:11.020 INFO:teuthology.orchestra.run.smithi038.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-19T05:29:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-19T05:29:11.080 INFO:teuthology.orchestra.run.smithi047.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T05:29:11.130 INFO:teuthology.orchestra.run.smithi047.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-09-19T05:29:11.172 INFO:teuthology.orchestra.run.smithi047.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-09-19T05:29:11.197 INFO:teuthology.orchestra.run.smithi047.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-19T05:29:11.314 INFO:teuthology.orchestra.run.smithi047.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-09-19T05:29:11.337 INFO:teuthology.orchestra.run.smithi038.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-09-19T05:29:11.356 INFO:teuthology.orchestra.run.smithi047.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-19T05:29:11.398 INFO:teuthology.orchestra.run.smithi047.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-19T05:29:11.425 INFO:teuthology.orchestra.run.smithi047.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 317 kB/s | 22 kB 00:00 2024-09-19T05:29:11.450 INFO:teuthology.orchestra.run.smithi047.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 313 kB 00:00 2024-09-19T05:29:11.475 INFO:teuthology.orchestra.run.smithi047.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-19T05:29:11.567 INFO:teuthology.orchestra.run.smithi047.stdout:(54/156): ceph-mgr-diskprediction-local-19.1.1- 3.3 MB/s | 7.4 MB 00:02 2024-09-19T05:29:11.584 INFO:teuthology.orchestra.run.smithi047.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-19T05:29:11.596 INFO:teuthology.orchestra.run.smithi038.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-19T05:29:11.643 INFO:teuthology.orchestra.run.smithi047.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-19T05:29:11.660 INFO:teuthology.orchestra.run.smithi047.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-19T05:29:11.663 INFO:teuthology.orchestra.run.smithi038.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T05:29:11.676 INFO:teuthology.orchestra.run.smithi047.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-19T05:29:11.702 INFO:teuthology.orchestra.run.smithi047.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-19T05:29:11.727 INFO:teuthology.orchestra.run.smithi047.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-19T05:29:11.744 INFO:teuthology.orchestra.run.smithi047.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-19T05:29:11.761 INFO:teuthology.orchestra.run.smithi047.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-19T05:29:11.789 INFO:teuthology.orchestra.run.smithi047.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-09-19T05:29:11.811 INFO:teuthology.orchestra.run.smithi047.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T05:29:11.837 INFO:teuthology.orchestra.run.smithi047.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-19T05:29:11.862 INFO:teuthology.orchestra.run.smithi047.stdout:(66/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-19T05:29:11.996 INFO:teuthology.orchestra.run.smithi047.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-19T05:29:11.996 INFO:teuthology.orchestra.run.smithi038.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-09-19T05:29:12.038 INFO:teuthology.orchestra.run.smithi047.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-09-19T05:29:12.071 INFO:teuthology.orchestra.run.smithi047.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T05:29:12.096 INFO:teuthology.orchestra.run.smithi047.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T05:29:12.163 INFO:teuthology.orchestra.run.smithi047.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T05:29:12.188 INFO:teuthology.orchestra.run.smithi038.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.0 MB/s | 4.9 MB 00:01 2024-09-19T05:29:12.197 INFO:teuthology.orchestra.run.smithi047.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-19T05:29:12.222 INFO:teuthology.orchestra.run.smithi047.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T05:29:12.247 INFO:teuthology.orchestra.run.smithi038.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T05:29:12.264 INFO:teuthology.orchestra.run.smithi047.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T05:29:12.339 INFO:teuthology.orchestra.run.smithi047.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T05:29:12.364 INFO:teuthology.orchestra.run.smithi047.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T05:29:12.390 INFO:teuthology.orchestra.run.smithi047.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T05:29:12.405 INFO:teuthology.orchestra.run.smithi038.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-09-19T05:29:12.415 INFO:teuthology.orchestra.run.smithi047.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-19T05:29:12.440 INFO:teuthology.orchestra.run.smithi047.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T05:29:12.447 INFO:teuthology.orchestra.run.smithi038.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T05:29:12.466 INFO:teuthology.orchestra.run.smithi047.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T05:29:12.499 INFO:teuthology.orchestra.run.smithi047.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-19T05:29:12.521 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-09-19T05:29:12.521 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-09-19T05:29:12.548 INFO:teuthology.orchestra.run.smithi038.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-19T05:29:12.592 INFO:teuthology.orchestra.run.smithi047.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-19T05:29:12.617 INFO:teuthology.orchestra.run.smithi047.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-19T05:29:12.642 INFO:teuthology.orchestra.run.smithi047.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T05:29:12.656 INFO:teuthology.orchestra.run.smithi038.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-19T05:29:12.667 INFO:teuthology.orchestra.run.smithi047.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T05:29:12.693 INFO:teuthology.orchestra.run.smithi047.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-19T05:29:12.698 INFO:teuthology.orchestra.run.smithi038.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-19T05:29:12.718 INFO:teuthology.orchestra.run.smithi047.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-19T05:29:12.740 INFO:teuthology.orchestra.run.smithi038.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-09-19T05:29:12.743 INFO:teuthology.orchestra.run.smithi047.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T05:29:12.777 INFO:teuthology.orchestra.run.smithi047.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T05:29:12.802 INFO:teuthology.orchestra.run.smithi047.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-09-19T05:29:12.815 INFO:teuthology.orchestra.run.smithi038.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-19T05:29:12.827 INFO:teuthology.orchestra.run.smithi047.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T05:29:12.858 INFO:teuthology.orchestra.run.smithi038.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-09-19T05:29:12.903 INFO:teuthology.orchestra.run.smithi047.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T05:29:12.928 INFO:teuthology.orchestra.run.smithi047.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-19T05:29:12.953 INFO:teuthology.orchestra.run.smithi047.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-19T05:29:12.978 INFO:teuthology.orchestra.run.smithi047.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T05:29:13.003 INFO:teuthology.orchestra.run.smithi047.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-19T05:29:13.029 INFO:teuthology.orchestra.run.smithi047.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T05:29:13.054 INFO:teuthology.orchestra.run.smithi047.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T05:29:13.071 INFO:teuthology.orchestra.run.smithi047.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-09-19T05:29:13.088 INFO:teuthology.orchestra.run.smithi047.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.8 MB/s | 96 kB 00:00 2024-09-19T05:29:13.105 INFO:teuthology.orchestra.run.smithi047.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-19T05:29:13.122 INFO:teuthology.orchestra.run.smithi047.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-19T05:29:13.222 INFO:teuthology.orchestra.run.smithi047.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-19T05:29:13.289 INFO:teuthology.orchestra.run.smithi047.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-19T05:29:13.448 INFO:teuthology.orchestra.run.smithi047.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-19T05:29:13.481 INFO:teuthology.orchestra.run.smithi047.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.6 MB/s | 308 kB 00:00 2024-09-19T05:29:13.507 INFO:teuthology.orchestra.run.smithi047.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-09-19T05:29:13.516 INFO:teuthology.orchestra.run.smithi038.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.1 MB/s | 5.8 MB 00:01 2024-09-19T05:29:13.540 INFO:teuthology.orchestra.run.smithi047.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-19T05:29:13.565 INFO:teuthology.orchestra.run.smithi047.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-19T05:29:13.599 INFO:teuthology.orchestra.run.smithi047.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T05:29:13.616 INFO:teuthology.orchestra.run.smithi038.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-19T05:29:13.624 INFO:teuthology.orchestra.run.smithi047.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 579 kB/s | 49 kB 00:00 2024-09-19T05:29:13.658 INFO:teuthology.orchestra.run.smithi047.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-19T05:29:13.700 INFO:teuthology.orchestra.run.smithi047.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T05:29:13.717 INFO:teuthology.orchestra.run.smithi038.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-19T05:29:13.725 INFO:teuthology.orchestra.run.smithi047.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T05:29:13.751 INFO:teuthology.orchestra.run.smithi047.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T05:29:13.775 INFO:teuthology.orchestra.run.smithi038.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T05:29:13.776 INFO:teuthology.orchestra.run.smithi047.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-19T05:29:13.817 INFO:teuthology.orchestra.run.smithi038.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T05:29:13.843 INFO:teuthology.orchestra.run.smithi047.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-19T05:29:13.868 INFO:teuthology.orchestra.run.smithi047.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-09-19T05:29:13.893 INFO:teuthology.orchestra.run.smithi047.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-09-19T05:29:13.918 INFO:teuthology.orchestra.run.smithi038.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-19T05:29:13.918 INFO:teuthology.orchestra.run.smithi047.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T05:29:13.944 INFO:teuthology.orchestra.run.smithi047.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-09-19T05:29:13.969 INFO:teuthology.orchestra.run.smithi047.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-19T05:29:13.993 INFO:teuthology.orchestra.run.smithi038.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-19T05:29:13.994 INFO:teuthology.orchestra.run.smithi047.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-19T05:29:14.020 INFO:teuthology.orchestra.run.smithi047.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-19T05:29:14.045 INFO:teuthology.orchestra.run.smithi047.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-19T05:29:14.070 INFO:teuthology.orchestra.run.smithi047.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T05:29:14.095 INFO:teuthology.orchestra.run.smithi047.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T05:29:14.121 INFO:teuthology.orchestra.run.smithi047.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-19T05:29:14.146 INFO:teuthology.orchestra.run.smithi047.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T05:29:14.160 INFO:teuthology.orchestra.run.smithi038.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-09-19T05:29:14.171 INFO:teuthology.orchestra.run.smithi047.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T05:29:14.185 INFO:teuthology.orchestra.run.smithi038.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-09-19T05:29:14.188 INFO:teuthology.orchestra.run.smithi047.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-19T05:29:14.210 INFO:teuthology.orchestra.run.smithi038.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T05:29:14.222 INFO:teuthology.orchestra.run.smithi047.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-19T05:29:14.235 INFO:teuthology.orchestra.run.smithi038.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T05:29:14.247 INFO:teuthology.orchestra.run.smithi047.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-19T05:29:14.277 INFO:teuthology.orchestra.run.smithi038.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T05:29:14.281 INFO:teuthology.orchestra.run.smithi047.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-19T05:29:14.306 INFO:teuthology.orchestra.run.smithi047.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T05:29:14.319 INFO:teuthology.orchestra.run.smithi038.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-19T05:29:14.331 INFO:teuthology.orchestra.run.smithi047.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T05:29:14.357 INFO:teuthology.orchestra.run.smithi047.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-19T05:29:14.407 INFO:teuthology.orchestra.run.smithi047.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-09-19T05:29:14.411 INFO:teuthology.orchestra.run.smithi038.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-09-19T05:29:14.432 INFO:teuthology.orchestra.run.smithi047.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-09-19T05:29:14.453 INFO:teuthology.orchestra.run.smithi038.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-09-19T05:29:14.482 INFO:teuthology.orchestra.run.smithi047.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-09-19T05:29:14.508 INFO:teuthology.orchestra.run.smithi047.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-19T05:29:14.520 INFO:teuthology.orchestra.run.smithi038.stdout:(96/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-19T05:29:14.533 INFO:teuthology.orchestra.run.smithi047.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 740 kB/s | 74 kB 00:00 2024-09-19T05:29:14.545 INFO:teuthology.orchestra.run.smithi038.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-09-19T05:29:14.558 INFO:teuthology.orchestra.run.smithi047.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-19T05:29:14.583 INFO:teuthology.orchestra.run.smithi047.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T05:29:14.587 INFO:teuthology.orchestra.run.smithi038.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T05:29:14.609 INFO:teuthology.orchestra.run.smithi047.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T05:29:14.634 INFO:teuthology.orchestra.run.smithi047.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T05:29:14.646 INFO:teuthology.orchestra.run.smithi038.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-19T05:29:14.659 INFO:teuthology.orchestra.run.smithi047.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T05:29:14.671 INFO:teuthology.orchestra.run.smithi038.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-19T05:29:14.710 INFO:teuthology.orchestra.run.smithi047.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T05:29:14.713 INFO:teuthology.orchestra.run.smithi038.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-19T05:29:14.738 INFO:teuthology.orchestra.run.smithi038.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-19T05:29:14.789 INFO:teuthology.orchestra.run.smithi038.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-19T05:29:14.839 INFO:teuthology.orchestra.run.smithi038.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T05:29:14.873 INFO:teuthology.orchestra.run.smithi038.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T05:29:14.889 INFO:teuthology.orchestra.run.smithi038.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-19T05:29:14.965 INFO:teuthology.orchestra.run.smithi038.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-19T05:29:15.010 INFO:teuthology.orchestra.run.smithi047.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-19T05:29:15.015 INFO:teuthology.orchestra.run.smithi038.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-19T05:29:15.074 INFO:teuthology.orchestra.run.smithi038.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-19T05:29:15.099 INFO:teuthology.orchestra.run.smithi038.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-19T05:29:15.124 INFO:teuthology.orchestra.run.smithi038.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T05:29:15.158 INFO:teuthology.orchestra.run.smithi038.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-19T05:29:15.200 INFO:teuthology.orchestra.run.smithi038.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T05:29:15.225 INFO:teuthology.orchestra.run.smithi038.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T05:29:15.250 INFO:teuthology.orchestra.run.smithi038.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T05:29:15.276 INFO:teuthology.orchestra.run.smithi038.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-19T05:29:15.277 INFO:teuthology.orchestra.run.smithi047.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-09-19T05:29:15.301 INFO:teuthology.orchestra.run.smithi038.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-19T05:29:15.326 INFO:teuthology.orchestra.run.smithi038.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T05:29:15.344 INFO:teuthology.orchestra.run.smithi047.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 4.6 MB/s | 3.2 MB 00:00 2024-09-19T05:29:15.352 INFO:teuthology.orchestra.run.smithi038.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-19T05:29:15.369 INFO:teuthology.orchestra.run.smithi047.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 446 kB/s | 41 kB 00:00 2024-09-19T05:29:15.377 INFO:teuthology.orchestra.run.smithi038.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T05:29:15.402 INFO:teuthology.orchestra.run.smithi038.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T05:29:15.427 INFO:teuthology.orchestra.run.smithi038.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T05:29:15.453 INFO:teuthology.orchestra.run.smithi038.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T05:29:15.478 INFO:teuthology.orchestra.run.smithi038.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-19T05:29:15.503 INFO:teuthology.orchestra.run.smithi038.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-19T05:29:15.529 INFO:teuthology.orchestra.run.smithi038.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-19T05:29:15.553 INFO:teuthology.orchestra.run.smithi047.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-09-19T05:29:15.554 INFO:teuthology.orchestra.run.smithi038.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 795 kB/s | 20 kB 00:00 2024-09-19T05:29:15.578 INFO:teuthology.orchestra.run.smithi047.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.0 MB/s | 249 kB 00:00 2024-09-19T05:29:15.621 INFO:teuthology.orchestra.run.smithi038.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-19T05:29:15.648 INFO:teuthology.orchestra.run.smithi038.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T05:29:15.671 INFO:teuthology.orchestra.run.smithi038.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T05:29:15.697 INFO:teuthology.orchestra.run.smithi038.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-19T05:29:15.739 INFO:teuthology.orchestra.run.smithi038.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-19T05:29:15.764 INFO:teuthology.orchestra.run.smithi038.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-19T05:29:15.789 INFO:teuthology.orchestra.run.smithi038.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-19T05:29:15.814 INFO:teuthology.orchestra.run.smithi038.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T05:29:15.840 INFO:teuthology.orchestra.run.smithi038.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T05:29:15.861 INFO:teuthology.orchestra.run.smithi047.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-09-19T05:29:15.865 INFO:teuthology.orchestra.run.smithi038.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-19T05:29:15.890 INFO:teuthology.orchestra.run.smithi038.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T05:29:15.915 INFO:teuthology.orchestra.run.smithi038.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-19T05:29:15.941 INFO:teuthology.orchestra.run.smithi038.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T05:29:15.975 INFO:teuthology.orchestra.run.smithi038.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-19T05:29:15.992 INFO:teuthology.orchestra.run.smithi038.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-19T05:29:16.017 INFO:teuthology.orchestra.run.smithi038.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T05:29:16.034 INFO:teuthology.orchestra.run.smithi038.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T05:29:16.068 INFO:teuthology.orchestra.run.smithi038.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T05:29:16.101 INFO:teuthology.orchestra.run.smithi038.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T05:29:16.160 INFO:teuthology.orchestra.run.smithi038.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-19T05:29:16.419 INFO:teuthology.orchestra.run.smithi038.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-19T05:29:16.469 INFO:teuthology.orchestra.run.smithi038.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-19T05:29:16.720 INFO:teuthology.orchestra.run.smithi047.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-19T05:29:16.726 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:29:16.726 INFO:teuthology.orchestra.run.smithi047.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-19T05:29:16.752 INFO:teuthology.orchestra.run.smithi038.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 7.0 MB/s | 2.3 MB 00:00 2024-09-19T05:29:16.794 INFO:teuthology.orchestra.run.smithi038.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 5.6 MB/s | 1.8 MB 00:00 2024-09-19T05:29:16.820 INFO:teuthology.orchestra.run.smithi038.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 613 kB/s | 41 kB 00:00 2024-09-19T05:29:16.882 INFO:teuthology.orchestra.run.smithi038.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.8 MB/s | 249 kB 00:00 2024-09-19T05:29:16.945 INFO:teuthology.orchestra.run.smithi038.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-19T05:29:17.029 INFO:teuthology.orchestra.run.smithi038.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-19T05:29:17.975 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-19T05:29:18.140 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-19T05:29:18.140 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-19T05:29:18.795 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-09-19T05:29:18.919 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:29:18.957 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:29:19.019 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:29:19.053 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:29:19.078 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:29:19.120 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:29:19.442 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:29:19.629 INFO:teuthology.orchestra.run.smithi038.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.0 MB/s | 16 MB 00:05 2024-09-19T05:29:19.634 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:29:19.634 INFO:teuthology.orchestra.run.smithi038.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-19T05:29:19.672 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:19.928 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:19.955 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:20.037 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-19T05:29:20.037 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-19T05:29:20.048 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:20.077 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:29:20.227 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:29:20.290 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:29:20.318 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:20.412 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:20.435 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:20.481 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:20.503 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:20.527 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:20.863 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-19T05:29:21.031 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-19T05:29:21.031 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-19T05:29:21.031 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:21.160 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:21.294 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:29:21.401 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:21.470 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:21.689 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:21.811 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:29:21.906 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:29:21.965 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:29:22.013 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:29:22.086 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:29:22.139 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:29:22.171 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:29:22.215 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:29:22.247 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:29:22.275 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:29:22.344 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:29:22.405 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:29:22.444 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:29:22.479 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:29:22.554 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:29:22.591 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:29:22.633 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:29:22.662 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:29:22.731 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:29:22.777 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:29:22.859 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:29:22.999 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:29:23.071 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:29:23.420 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-19T05:29:23.420 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-19T05:29:23.589 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:29:23.674 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:29:23.704 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:29:23.738 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:29:23.785 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:29:23.829 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:29:23.972 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:29:24.033 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:29:24.088 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:29:24.160 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:29:24.192 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:29:24.228 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:29:24.283 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:29:24.328 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:29:24.370 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:29:24.412 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:29:24.548 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:29:24.589 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:29:24.630 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:29:24.734 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:29:24.773 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:29:24.813 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:29:24.938 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:29:24.963 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:29:25.037 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:29:25.162 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:29:25.195 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:29:25.223 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:29:25.257 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:29:25.296 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:29:25.336 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:29:25.391 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:29:25.426 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:29:25.460 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:29:25.500 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:29:25.535 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:29:25.620 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:29:25.673 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:29:25.705 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:29:25.806 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:29:25.864 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:29:26.581 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:29:26.644 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:29:26.678 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:29:26.775 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:29:26.805 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-19T05:29:26.825 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:29:26.878 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:29:26.915 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:29:26.954 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:29:26.955 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:29:26.990 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:29:27.008 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:29:27.042 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:29:27.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:29:27.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:29:27.172 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:29:27.204 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:29:27.289 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:29:27.458 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:29:27.551 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:27.659 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:27.685 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:27.806 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:29:27.925 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:29:27.963 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:29:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:29:28.034 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:29:28.048 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:28.145 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:28.182 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:28.259 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:28.282 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:28.566 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:28.690 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:29:28.779 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:28.850 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:29.067 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:29.176 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:29:29.245 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:29:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:29:29.313 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:29:29.324 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:29:29.352 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:29:29.357 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:29:29.385 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:29:29.432 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:29:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:29:29.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:29:29.567 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:29:29.599 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:29:29.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:29:29.693 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:29:29.698 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:29:29.740 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:29.759 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:29:29.788 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:29:29.832 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:29:29.835 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-19T05:29:29.837 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:29.878 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:29:29.911 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:29:29.914 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:29:29.945 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:29:29.958 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:29:29.990 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:29:30.005 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:29:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:29:30.059 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:29:30.092 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:29:30.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:29:30.117 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:29:30.141 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:29:30.172 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:29:30.218 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:29:30.356 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:29:30.419 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:29:30.479 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:29:30.510 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:30.567 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:29:30.580 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:30.611 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:29:30.677 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:30.801 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:29:30.828 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:30.920 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:29:30.949 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:29:30.976 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:29:31.058 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:29:31.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:29:31.117 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:29:31.127 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:29:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:29:31.171 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:29:31.198 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:31.199 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:29:31.305 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:29:31.333 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:31.368 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:29:31.379 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:29:31.401 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:31.418 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:29:31.425 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:29:31.475 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:29:31.549 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:29:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:29:31.621 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:31.622 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:29:31.670 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:29:31.705 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:29:31.748 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:29:31.779 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:29:31.790 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:29:31.898 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:29:31.933 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:29:31.978 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:29:32.005 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:32.020 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:29:32.083 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:32.127 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:29:32.162 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:29:32.201 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:29:32.227 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:29:32.251 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:29:32.310 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:29:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:29:32.416 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:29:32.458 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:29:32.520 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:29:32.556 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:29:32.591 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:29:32.668 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:29:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:29:32.716 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:29:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:29:32.757 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:29:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:29:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:29:32.841 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:29:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:29:32.868 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:29:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:29:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:29:32.939 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:32.940 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:29:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:29:32.974 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:29:32.992 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:29:33.022 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:29:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:29:33.099 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:29:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:29:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:29:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:29:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:29:33.345 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:29:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:29:33.431 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:29:33.481 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:29:33.509 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:29:33.518 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:33.561 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:33.590 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:29:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:29:33.652 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:33.652 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:29:33.652 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:29:33.652 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:33.687 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:33.716 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:29:33.862 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:29:33.917 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:29:34.128 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:29:34.175 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:29:34.209 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:29:34.314 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:29:34.365 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:29:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:29:34.469 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:29:34.493 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:29:34.528 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:29:34.563 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:29:34.598 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:29:34.641 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:29:34.679 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:29:34.720 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:29:34.721 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:29:34.751 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:29:34.839 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:29:34.892 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:29:34.940 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:29:34.996 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:29:35.069 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:29:35.107 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:29:35.144 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:29:35.191 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:29:35.226 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:29:35.270 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:29:35.312 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:29:35.455 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:29:35.513 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:29:35.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:29:35.554 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:29:35.637 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:29:35.661 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:29:35.730 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:29:35.770 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:29:35.802 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:29:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:29:35.940 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:29:36.076 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:29:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:29:36.155 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:29:36.189 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:29:36.232 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:29:36.271 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:29:36.301 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:29:36.336 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:29:36.369 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:29:36.411 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:29:36.445 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:29:36.543 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:29:36.601 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:29:36.631 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:29:36.747 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:29:36.800 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:29:36.942 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:29:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:29:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:29:37.074 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:29:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:29:37.445 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:37.537 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:37.560 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:29:37.566 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:29:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:29:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:29:37.647 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:29:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:29:37.802 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:29:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:29:37.938 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:29:37.983 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:29:38.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:29:38.139 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:38.174 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:29:38.211 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:38.213 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:29:38.293 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:29:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:29:39.116 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:29:40.444 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:29:40.520 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:29:40.548 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:29:40.581 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:29:40.655 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:40.696 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:40.908 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:29:40.960 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:41.060 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:29:41.090 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:29:41.126 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:29:41.272 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:41.308 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:41.401 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:41.401 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:29:41.401 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:29:41.401 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:41.434 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:41.636 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:29:41.663 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:41.735 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:29:41.765 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:29:44.141 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:44.186 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:44.777 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:29:44.814 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:44.889 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:29:44.889 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:29:44.889 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:29:44.889 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:44.932 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:45.730 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:45.730 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-09-19T05:29:45.730 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-09-19T05:29:45.730 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-09-19T05:29:45.730 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-09-19T05:29:45.731 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-09-19T05:29:45.731 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-09-19T05:29:45.731 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-09-19T05:29:45.731 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:45.908 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:45.924 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:45.925 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:45.925 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:29:45.925 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:47.438 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:47.438 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:47.438 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:29:47.439 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:47.439 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:47.439 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:47.661 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:47.685 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:47.686 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:47.686 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:29:47.686 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:47.686 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:47.686 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:48.129 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:48.153 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:48.258 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:29:48.315 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:29:48.371 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:29:48.562 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:29:48.613 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:29:48.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:29:48.771 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:29:48.917 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:29:48.972 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:29:49.003 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:29:49.031 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:29:49.536 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:29:49.563 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:29:49.603 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:29:49.673 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:29:49.711 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:29:50.750 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:29:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:29:51.745 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:29:51.780 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:29:51.883 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:29:52.053 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:29:52.082 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:29:52.106 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:29:52.107 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:52.107 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:29:52.107 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:29:52.107 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:29:52.107 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:52.158 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:29:52.170 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:29:52.267 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:29:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:29:53.256 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:29:53.286 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:29:53.309 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:29:53.309 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:53.309 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:29:53.309 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:29:53.310 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:29:53.310 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:53.353 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:29:53.376 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:29:53.376 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:53.376 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:29:53.376 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:53.429 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:29:53.452 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:29:53.452 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:53.452 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:29:53.452 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:29:53.452 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:29:53.453 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:53.754 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-09-19T05:29:53.755 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:29:53.780 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:29:53.930 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:53.946 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:53.946 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:53.946 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:29:53.946 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:55.483 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:55.509 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:55.509 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:55.509 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:29:55.509 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:55.510 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:55.510 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:55.750 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:55.773 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:56.234 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:56.257 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:56.257 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:56.257 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:29:56.258 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:56.258 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:56.258 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:29:56.365 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:29:56.424 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:29:56.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:29:56.691 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:29:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:29:56.818 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:29:56.915 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:29:57.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:29:57.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:29:57.136 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:29:57.136 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-09-19T05:29:57.137 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:29:57.170 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:29:57.318 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:57.335 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:29:57.335 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:57.335 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:29:57.336 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:57.633 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:29:57.660 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:29:57.692 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:29:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:29:57.783 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:29:58.350 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:29:58.391 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:29:58.432 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:29:58.472 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:29:58.508 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:29:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:29:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:29:58.760 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:29:58.798 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:29:58.844 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:29:58.863 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:29:58.892 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:58.900 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:29:58.935 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:29:59.130 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:59.155 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:29:59.155 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:59.155 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:29:59.155 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:59.156 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:29:59.156 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:59.413 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:29:59.450 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:29:59.471 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:29:59.615 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:29:59.640 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:29:59.754 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:29:59.811 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:29:59.876 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:29:59.951 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:29:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:30:00.070 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:30:00.086 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:30:00.124 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:30:00.204 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:30:00.263 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:30:00.287 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:30:00.292 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:30:00.295 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:30:00.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:00.380 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:30:00.388 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:30:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:30:00.420 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:30:00.439 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:30:00.490 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:30:00.501 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:30:00.502 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:30:00.523 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:30:00.541 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:30:00.550 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:30:00.553 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:30:00.566 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:30:00.593 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:30:00.593 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:00.677 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:00.677 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:00.769 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:00.803 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:30:00.808 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:00.808 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:01.004 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:01.005 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:01.048 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:30:01.082 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:30:01.088 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:01.114 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:30:01.151 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:01.187 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:30:01.213 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:30:01.532 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:30:01.557 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:30:01.581 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:30:01.581 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:01.581 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:30:01.581 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:30:01.582 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:30:01.582 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:01.624 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:30:01.647 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:30:01.647 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:01.647 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:30:01.647 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:01.700 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:30:01.723 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:02.042 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:30:02.065 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:30:02.065 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:02.066 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:30:02.066 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:30:02.066 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:30:02.066 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:02.322 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:30:02.357 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:30:03.373 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:30:03.399 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:30:03.509 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:30:03.677 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:30:03.709 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:30:03.735 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:03.793 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:30:03.805 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:30:03.919 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:30:03.972 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:30:04.955 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:30:04.988 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:30:05.012 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:30:05.012 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:05.012 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:30:05.013 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:30:05.013 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:30:05.013 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:05.055 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:30:05.079 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:30:05.080 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:05.080 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:30:05.080 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:05.130 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:30:05.155 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:05.473 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:30:05.497 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:30:05.498 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:30:05.498 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:30:05.498 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:30:05.498 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:30:05.498 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:30:06.745 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:30:06.746 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:30:06.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:30:06.748 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:30:06.761 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:30:06.761 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:30:06.761 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:30:06.762 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:30:06.763 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:30:06.764 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:30:06.765 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:30:06.766 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:30:06.767 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:30:06.768 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:30:06.769 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:30:06.771 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:30:06.772 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:30:06.773 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:30:06.774 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:30:06.775 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:30:06.775 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:30:06.775 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:30:06.775 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:30:06.776 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:30:06.776 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:30:06.776 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:30:06.776 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:30:06.776 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:30:06.777 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:30:06.778 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:30:06.779 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:30:06.779 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:30:06.779 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:30:06.779 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:30:06.780 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:30:06.780 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:30:06.780 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:30:06.780 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:30:06.780 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:30:06.781 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:30:06.782 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:30:06.782 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:30:06.880 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:30:06.921 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:30:06.963 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:30:06.994 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:30:07.021 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:30:07.219 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:30:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:30:07.290 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:30:07.319 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:30:07.357 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:30:07.997 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:08.038 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:08.059 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:08.972 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:30:09.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:30:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:30:09.194 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:30:09.229 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:30:09.255 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:30:09.289 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:30:09.289 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:09.373 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:09.373 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:09.449 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:09.474 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:30:09.479 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:09.479 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:09.514 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:09.514 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:30:09.515 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-09-19T05:30:09.515 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:09.515 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:09.515 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:09.515 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:30:09.516 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.517 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.518 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.519 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:30:09.520 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.521 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:30:09.522 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:30:09.523 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:30:09.524 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:30:09.525 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:30:09.526 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:30:09.527 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:30:09.528 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:30:09.529 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:30:09.530 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:30:09.531 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:30:09.532 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:30:09.535 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:30:09.535 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:30:09.535 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-09-19T05:30:09.660 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:09.660 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:09.743 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:09.817 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:09.917 DEBUG:teuthology.parallel:result is None 2024-09-19T05:30:10.335 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:30:10.377 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:30:10.417 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:30:10.449 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:30:10.476 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:30:10.684 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:30:10.731 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:30:10.780 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:30:10.808 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:30:10.845 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:30:11.492 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:11.535 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:11.556 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:30:12.475 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:30:12.573 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:30:12.620 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:30:12.709 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:30:12.743 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:30:12.768 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:30:12.802 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:30:12.803 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:12.878 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:30:12.878 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:12.954 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:30:12.978 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:30:12.984 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:12.984 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:13.173 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:30:13.173 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:13.248 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:30:13.317 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:30:15.158 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:30:15.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:30:15.160 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:30:15.161 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:30:15.162 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:30:15.163 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:30:15.164 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:30:15.165 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:30:15.166 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:30:15.167 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:30:15.168 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:30:15.169 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:30:15.170 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:30:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:30:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:30:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:30:15.171 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:30:18.159 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-19T05:30:18.160 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.161 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.162 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:30:18.163 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.164 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:18.165 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:30:18.166 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:30:18.167 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:30:18.168 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:30:18.169 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:30:18.170 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:30:18.171 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:30:18.172 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:30:18.173 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:30:18.173 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:30:18.173 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-19T05:30:18.550 DEBUG:teuthology.parallel:result is None 2024-09-19T05:30:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:30:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:30:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:30:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:30:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:30:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:30:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:30:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:30:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:30:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:30:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:30:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:30:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:30:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:30:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:30:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:30:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:30:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:30:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:30:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:30:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:30:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:30:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:30:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:30:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:30:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:30:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:30:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:30:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:30:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:30:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:30:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:30:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:30:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:30:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:30:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:30:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:30:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:30:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:30:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:30:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:30:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:30:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:30:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:30:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:30:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:30:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:30:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:30:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:30:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:30:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:30:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:30:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:30:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:30:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:30:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:30:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:30:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:30:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:30:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:30:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:30:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:30:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:30:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:30:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:30:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:30:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:30:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:30:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:30:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:30:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:30:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:30:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:30:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:30:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:30:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:30:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:30:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:30:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:30:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:30:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:30:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:30:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:30:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:30:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:30:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:30:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:30:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:30:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:30:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:30:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:30:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:30:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:30:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:30:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:30:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:30:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:30:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:30:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:30:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:30:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:30:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:30:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:30:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:30:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:30:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:30:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:21.764 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:30:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:30:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:30:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:30:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:30:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:30:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:30:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:30:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:30:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:30:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:30:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:30:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:30:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:30:21.777 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-19T05:30:22.296 DEBUG:teuthology.parallel:result is None 2024-09-19T05:30:22.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:22.448 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:30:22.472 INFO:teuthology.orchestra.run.smithi038.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.473 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.473 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:30:22.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:22.616 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:30:22.642 INFO:teuthology.orchestra.run.smithi047.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.642 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.642 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:30:22.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:22.782 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:30:22.810 INFO:teuthology.orchestra.run.smithi179.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.810 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:30:22.810 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:30:22.812 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T05:30:22.813 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:30:22.813 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:30:22.845 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:30:22.845 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:30:22.877 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:30:22.878 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:30:22.911 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T05:30:22.911 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:30:22.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:30:22.939 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:30:23.007 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:30:23.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:30:23.037 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:30:23.106 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:30:23.106 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:30:23.137 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:30:23.223 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T05:30:23.224 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:30:23.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:30:23.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:30:23.322 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:30:23.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:30:23.351 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:30:23.421 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:30:23.421 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:30:23.446 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:30:23.512 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T05:30:23.512 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:30:23.513 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:30:23.541 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:30:23.609 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:30:23.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:30:23.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:30:23.709 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:30:23.709 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:30:23.733 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:30:23.798 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T05:30:23.905 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T05:30:23.905 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:23.906 INFO:tasks.cephadm:Cluster fsid is 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:30:23.906 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T05:30:23.906 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.38', 'mon.b': '172.21.15.47', 'mon.c': '172.21.15.179'} 2024-09-19T05:30:23.907 INFO:tasks.cephadm:First mon is mon.a on smithi038 2024-09-19T05:30:23.907 INFO:tasks.cephadm:First mgr is x 2024-09-19T05:30:23.907 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T05:30:23.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-09-19T05:30:23.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-09-19T05:30:23.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-09-19T05:30:23.991 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T05:30:23.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:24.129 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T05:30:24.266 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:30:24.268 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T05:30:24.268 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T05:30:24.269 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:24.652 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:30 /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:24.652 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.053 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:30 /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.053 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.427 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 05:30 /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.428 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.448 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.468 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T05:30:25.500 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T05:30:25.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:30:25.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:30:25.509 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T05:30:26.965 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:30:27.237 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:30:27.285 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:30:53.987 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-19T05:30:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-19T05:30:59.850 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-09-19T05:30:59.851 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-09-19T05:31:03.026 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:31:03.027 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:31:03.054 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-09-19T05:31:03.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-09-19T05:31:03.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-09-19T05:31:03.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-09-19T05:31:03.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-09-19T05:31:03.217 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-09-19T05:31:03.247 INFO:tasks.cephadm:Writing seed config... 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T05:31:03.248 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T05:31:03.249 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T05:31:03.250 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T05:31:03.251 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T05:31:03.252 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:31:03.252 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T05:31:03.269 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 44ff0e58-7648-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T05:31:03.269 DEBUG:teuthology.orchestra.run.smithi038:mon.a> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T05:31:03.312 DEBUG:teuthology.orchestra.run.smithi038:mgr.x> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T05:31:03.355 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T05:31:03.355 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:31:03.565 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:31:03.565 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '44ff0e58-7648-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-09-19T05:31:03.565 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T05:31:03.566 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-09-19T05:31:03.606 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-19T05:31:03.606 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-09-19T05:31:03.606 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-09-19T05:31:03.614 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T05:31:03.614 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T05:31:03.620 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T05:31:03.621 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:31:03.627 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-19T05:31:03.634 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-19T05:31:03.634 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-19T05:31:03.634 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-09-19T05:31:03.656 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-19T05:31:03.657 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T05:31:03.657 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-09-19T05:31:03.657 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-09-19T05:31:03.664 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T05:31:03.664 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T05:31:03.670 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T05:31:03.671 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:31:03.677 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-19T05:31:03.684 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-19T05:31:03.684 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-19T05:31:03.684 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-09-19T05:31:03.684 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:03.685 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 140650055951360 on /run/cephadm/44ff0e58-7648-11ef-bceb-c7b262605968.lock 2024-09-19T05:31:03.685 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 140650055951360 acquired on /run/cephadm/44ff0e58-7648-11ef-bceb-c7b262605968.lock 2024-09-19T05:31:03.685 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-09-19T05:31:03.685 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-09-19T05:31:03.685 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-19T05:31:03.689 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-09-19T05:31:03.689 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-09-19T05:31:03.691 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T05:31:03.691 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T05:31:03.691 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-19T05:31:03.694 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T05:31:03.695 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-19T05:31:03.695 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-19T05:31:03.695 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T05:31:03.695 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T05:31:03.696 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T05:31:06.877 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T05:31:16.175 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T05:31:16.176 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T05:31:16.176 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T05:31:17.148 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-09-19T05:31:17.148 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-09-19T05:31:17.987 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQAlt+tmGwyqGhAAsO2YzVjLAb3a3+d58gkLyA== 2024-09-19T05:31:18.843 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQAmt+tm93wpERAAVRylpPBBanyYyNkPsJIBTA== 2024-09-19T05:31:19.799 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQAnt+tm1Ax/CxAAGCgGfYixarY1DeRVn1hkmg== 2024-09-19T05:31:19.800 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for a [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T05:31:20.742 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:31:20.742 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-09-19T05:31:22.126 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.a on 2024-09-19T05:31:22.353 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T05:31:22.506 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T05:31:22.726 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target. 2024-09-19T05:31:22.726 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target. 2024-09-19T05:31:22.977 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a 2024-09-19T05:31:22.978 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T05:31:23.156 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target.wants/ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968@.service. 2024-09-19T05:31:23.733 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:31:23.734 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-19T05:31:23.741 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:31:23.741 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:31:23.741 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-19T05:31:23.741 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T05:31:23.741 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-09-19T05:31:23.742 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-09-19T05:31:23.845 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:23 smithi038 systemd[1]: Started Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: monmap epoch 1 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: last_changed 2024-09-19T05:31:20.164217+0000 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: min_mon_release 19 (squid) 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: election_strategy: 1 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:31:24.935 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: fsmap 2024-09-19T05:31:24.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T05:31:24.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: mgrmap e1: no daemons active 2024-09-19T05:31:24.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:24 smithi038 ceph-mon[30858]: from='client.? 172.21.15.38:0/954135358' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.538431s) 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T05:31:24.944 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.150+0000 7f12ebea1640 1 Processor -- start 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.150+0000 7f12ebea1640 1 -- start start 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.150+0000 7f12ebea1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.150+0000 7f12ebea1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410a880 con 0x7f12e4107f50 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.151+0000 7f12e9c3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:24.945 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.151+0000 7f12e9c3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49722/0 (socket says 172.21.15.38:49722) 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.151+0000 7f12e9c3d640 1 -- 172.21.15.38:0/811901768 learned_addr learned my addr 172.21.15.38:0/811901768 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.151+0000 7f12e9c3d640 1 -- 172.21.15.38:0/811901768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e410aa00 con 0x7f12e4107f50 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12e9c3d640 1 --2- 172.21.15.38:0/811901768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009e80 tx=0x7f12d402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1bee391efaba12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12e8c3b640 1 -- 172.21.15.38:0/811901768 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d4036070 con 0x7f12e4107f50 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12e8c3b640 1 -- 172.21.15.38:0/811901768 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d4031400 con 0x7f12e4107f50 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12e8c3b640 1 -- 172.21.15.38:0/811901768 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d4031700 con 0x7f12e4107f50 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12ebea1640 1 -- 172.21.15.38:0/811901768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 msgr2=0x7f12e410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.153+0000 7f12ebea1640 1 --2- 172.21.15.38:0/811901768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009e80 tx=0x7f12d402a7a0 comp rx=0 tx=0).stop 2024-09-19T05:31:24.946 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- 172.21.15.38:0/811901768 shutdown_connections 2024-09-19T05:31:24.947 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 --2- 172.21.15.38:0/811901768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e410a340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:24.947 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- 172.21.15.38:0/811901768 >> 172.21.15.38:0/811901768 conn(0x7f12e4101af0 msgr2=0x7f12e4103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:24.947 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- 172.21.15.38:0/811901768 shutdown_connections 2024-09-19T05:31:24.947 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- 172.21.15.38:0/811901768 wait complete. 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 Processor -- start 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- start start 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.154+0000 7f12ebea1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410ae80 con 0x7f12e4107f50 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e9c3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e9c3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49734/0 (socket says 172.21.15.38:49734) 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e9c3d640 1 -- 172.21.15.38:0/954135358 learned_addr learned my addr 172.21.15.38:0/954135358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e9c3d640 1 -- 172.21.15.38:0/954135358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e419b2d0 con 0x7f12e4107f50 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e9c3d640 1 --2- 172.21.15.38:0/954135358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12d4034040 tx=0x7f12d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:24.948 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d4041070 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d403c650 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e419b560 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d4036040 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.155+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e419e210 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.156+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12d4044070 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.156+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f12d4046c10 con 0x7f12e4107f50 2024-09-19T05:31:24.949 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.156+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e41083b0 con 0x7f12e4107f50 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.159+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f12d403c950 con 0x7f12e4107f50 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.223+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f12e402d900 con 0x7f12e4107f50 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.223+0000 7f12e2ffd640 1 -- 172.21.15.38:0/954135358 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f12d403cb30 con 0x7f12e4107f50 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 msgr2=0x7f12e419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 --2- 172.21.15.38:0/954135358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12d4034040 tx=0x7f12d4004a40 comp rx=0 tx=0).stop 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 shutdown_connections 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 --2- 172.21.15.38:0/954135358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12e4107f50 0x7f12e419ad90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 >> 172.21.15.38:0/954135358 conn(0x7f12e4101af0 msgr2=0x7f12e4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 shutdown_connections 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:24.224+0000 7f12ebea1640 1 -- 172.21.15.38:0/954135358 wait complete. 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-09-19T05:31:24.950 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T05:31:26.000 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:25 smithi038 ceph-mon[30858]: from='client.? 172.21.15.38:0/732271010' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T05:31:26.000 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:25 smithi038 ceph-mon[30858]: from='client.? 172.21.15.38:0/732271010' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T05:31:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T05:31:26.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef29f6f640 1 Processor -- start 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef29f6f640 1 -- start start 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef29f6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef29f6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fef24107ee0 con 0x7fef24107510 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49742/0 (socket says 172.21.15.38:49742) 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.351+0000 7fef237fe640 1 -- 172.21.15.38:0/3491459095 learned_addr learned my addr 172.21.15.38:0/3491459095 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.352+0000 7fef237fe640 1 -- 172.21.15.38:0/3491459095 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241086c0 con 0x7fef24107510 2024-09-19T05:31:26.010 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.352+0000 7fef237fe640 1 --2- 172.21.15.38:0/3491459095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fef14009820 tx=0x7fef1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84d5725d9f773d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.352+0000 7fef227fc640 1 -- 172.21.15.38:0/3491459095 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef14037070 con 0x7fef24107510 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef227fc640 1 -- 172.21.15.38:0/3491459095 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef14032440 con 0x7fef24107510 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef227fc640 1 -- 172.21.15.38:0/3491459095 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef14032740 con 0x7fef24107510 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 -- 172.21.15.38:0/3491459095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 msgr2=0x7fef24107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 --2- 172.21.15.38:0/3491459095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fef14009820 tx=0x7fef1402a270 comp rx=0 tx=0).stop 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 -- 172.21.15.38:0/3491459095 shutdown_connections 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 --2- 172.21.15.38:0/3491459095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef24107910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 -- 172.21.15.38:0/3491459095 >> 172.21.15.38:0/3491459095 conn(0x7fef24102ce0 msgr2=0x7fef24105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 -- 172.21.15.38:0/3491459095 shutdown_connections 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.353+0000 7fef29f6f640 1 -- 172.21.15.38:0/3491459095 wait complete. 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef29f6f640 1 Processor -- start 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef29f6f640 1 -- start start 2024-09-19T05:31:26.011 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef29f6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef29f6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fef24108c60 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49746/0 (socket says 172.21.15.38:49746) 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.354+0000 7fef237fe640 1 -- 172.21.15.38:0/732271010 learned_addr learned my addr 172.21.15.38:0/732271010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef237fe640 1 -- 172.21.15.38:0/732271010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2419b1a0 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef237fe640 1 --2- 172.21.15.38:0/732271010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef140090b0 tx=0x7fef140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef14041070 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1403c7a0 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef14037040 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2419b430 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2419e120 con 0x7fef24107510 2024-09-19T05:31:26.012 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef14044070 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.355+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef1403d3f0 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.356+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef241092b0 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.358+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fef14046450 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.417+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fef2402d980 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.421+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fef14046630 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.421+0000 7fef20ff9640 1 -- 172.21.15.38:0/732271010 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef14056070 con 0x7fef24107510 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 msgr2=0x7fef2419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 --2- 172.21.15.38:0/732271010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef140090b0 tx=0x7fef140046c0 comp rx=0 tx=0).stop 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 shutdown_connections 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 --2- 172.21.15.38:0/732271010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef24107510 0x7fef2419ac60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 >> 172.21.15.38:0/732271010 conn(0x7fef24102ce0 msgr2=0x7fef2418ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 shutdown_connections 2024-09-19T05:31:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:25.422+0000 7fef29f6f640 1 -- 172.21.15.38:0/732271010 wait complete. 2024-09-19T05:31:26.014 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-09-19T05:31:27.015 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:26 smithi038 ceph-mon[30858]: from='client.? 172.21.15.38:0/2102038284' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:31:27.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.401+0000 7ff21ead5640 1 Processor -- start 2024-09-19T05:31:27.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.401+0000 7ff21ead5640 1 -- start start 2024-09-19T05:31:27.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.401+0000 7ff21ead5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:27.022 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.401+0000 7ff21ead5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810a880 con 0x7ff218107f50 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.402+0000 7ff21c871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.402+0000 7ff21c871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49750/0 (socket says 172.21.15.38:49750) 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.402+0000 7ff21c871640 1 -- 172.21.15.38:0/4291164093 learned_addr learned my addr 172.21.15.38:0/4291164093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.402+0000 7ff21c871640 1 -- 172.21.15.38:0/4291164093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21810aa00 con 0x7ff218107f50 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21c871640 1 --2- 172.21.15.38:0/4291164093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009820 tx=0x7ff20c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c71555617848c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff2177fe640 1 -- 172.21.15.38:0/4291164093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20c037070 con 0x7ff218107f50 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff2177fe640 1 -- 172.21.15.38:0/4291164093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff20c032900 con 0x7ff218107f50 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff2177fe640 1 -- 172.21.15.38:0/4291164093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20c030850 con 0x7ff218107f50 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 -- 172.21.15.38:0/4291164093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 msgr2=0x7ff21810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 --2- 172.21.15.38:0/4291164093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009820 tx=0x7ff20c02a270 comp rx=0 tx=0).stop 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 -- 172.21.15.38:0/4291164093 shutdown_connections 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 --2- 172.21.15.38:0/4291164093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff21810a340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:27.023 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 -- 172.21.15.38:0/4291164093 >> 172.21.15.38:0/4291164093 conn(0x7ff218101af0 msgr2=0x7ff218103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.403+0000 7ff21ead5640 1 -- 172.21.15.38:0/4291164093 shutdown_connections 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21ead5640 1 -- 172.21.15.38:0/4291164093 wait complete. 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21ead5640 1 Processor -- start 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21ead5640 1 -- start start 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21ead5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21ead5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810b9e0 con 0x7ff218107f50 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21c871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21c871640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49766/0 (socket says 172.21.15.38:49766) 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.404+0000 7ff21c871640 1 -- 172.21.15.38:0/2102038284 learned_addr learned my addr 172.21.15.38:0/2102038284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff21c871640 1 -- 172.21.15.38:0/2102038284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff218107d10 con 0x7ff218107f50 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff21c871640 1 --2- 172.21.15.38:0/2102038284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0097f0 tx=0x7ff20c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20c037040 con 0x7ff218107f50 2024-09-19T05:31:27.024 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff218105f00 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff20c040070 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff20c03a390 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181063e0 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff20c03a630 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.405+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff20c0459a0 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.406+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2181083b0 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.408+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff20c045460 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.469+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff218106660 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff215ffb640 1 -- 172.21.15.38:0/2102038284 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff20c03b430 con 0x7ff218107f50 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 msgr2=0x7ff2181077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff21ead5640 1 --2- 172.21.15.38:0/2102038284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0097f0 tx=0x7ff20c030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 shutdown_connections 2024-09-19T05:31:27.025 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff21ead5640 1 --2- 172.21.15.38:0/2102038284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff218107f50 0x7ff2181077d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:27.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.470+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 >> 172.21.15.38:0/2102038284 conn(0x7ff218101af0 msgr2=0x7ff2181026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:27.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.471+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 shutdown_connections 2024-09-19T05:31:27.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:26.471+0000 7ff21ead5640 1 -- 172.21.15.38:0/2102038284 wait complete. 2024-09-19T05:31:27.026 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-09-19T05:31:27.294 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 systemd[1]: Stopping Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:31:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a[30853]: 2024-09-19T05:31:27.292+0000 7f1c06caa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:31:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a[30853]: 2024-09-19T05:31:27.292+0000 7f1c06caa640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T05:31:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 podman[31100]: 2024-09-19 05:31:27.435724637 +0000 UTC m=+0.248200404 container died 263df1c5b206addecf6b2e34e7cb2411d32542ba62abb4b36c3d70bf3d865d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:31:27.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 podman[31100]: 2024-09-19 05:31:27.758670858 +0000 UTC m=+0.571146625 container cleanup 263df1c5b206addecf6b2e34e7cb2411d32542ba62abb4b36c3d70bf3d865d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T05:31:27.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 bash[31100]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a 2024-09-19T05:31:28.186 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:27 smithi038 podman[31112]: 2024-09-19 05:31:27.934858661 +0000 UTC m=+0.496354054 container remove 263df1c5b206addecf6b2e34e7cb2411d32542ba62abb4b36c3d70bf3d865d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:31:28.187 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T05:31:28.187 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 systemd[1]: Stopped Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:31:28.187 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 systemd[1]: Starting Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:31:28.476 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 podman[31208]: 2024-09-19 05:31:28.385773809 +0000 UTC m=+0.104416413 container create edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:31:28.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 podman[31208]: 2024-09-19 05:31:28.297614475 +0000 UTC m=+0.016257099 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:31:28.669 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T05:31:28.884 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 podman[31208]: 2024-09-19 05:31:28.571310991 +0000 UTC m=+0.289953596 container init edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 podman[31208]: 2024-09-19 05:31:28.574470898 +0000 UTC m=+0.293113503 container start edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: pidfile_write: ignore empty --pid-file 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: load: jerasure load: lrc 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: RocksDB version: 7.9.2 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Git sha 0 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: DB SUMMARY 2024-09-19T05:31:28.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: DB Session ID: 307FMT41WAZG1MNL4CJE 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: CURRENT file: CURRENT 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104248 ; 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.error_if_exists: 0 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.create_if_missing: 0 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.paranoid_checks: 1 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T05:31:28.886 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T05:31:28.887 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.env: 0x556057f39940 2024-09-19T05:31:28.887 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T05:31:28.887 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.info_log: 0x556059c196e0 2024-09-19T05:31:28.887 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T05:31:28.887 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.statistics: (nil) 2024-09-19T05:31:28.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.use_fsync: 0 2024-09-19T05:31:28.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_log_file_size: 0 2024-09-19T05:31:28.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T05:31:28.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T05:31:28.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T05:31:28.890 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T05:31:28.890 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_fallocate: 1 2024-09-19T05:31:28.890 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.use_direct_reads: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.db_log_dir: 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.wal_dir: 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T05:31:28.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.write_buffer_manager: 0x556059c28dc0 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T05:31:28.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.unordered_write: 0 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.row_cache: None 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.wal_filter: None 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T05:31:28.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.two_write_queues: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.wal_compression: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.atomic_flush: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.log_readahead_size: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T05:31:28.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_background_jobs: 2 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_background_compactions: -1 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_subcompactions: 1 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T05:31:28.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_open_files: -1 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T05:31:28.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_background_flushes: -1 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Compression algorithms supported: 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kZSTD supported: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kXpressCompression supported: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kBZip2Compression supported: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kLZ4Compression supported: 1 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kZlibCompression supported: 1 2024-09-19T05:31:28.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: kSnappyCompression supported: 1 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.merge_operator: 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_filter: None 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T05:31:28.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556059c19320) 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-09-19T05:31:28.899 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_type: 0 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_index_type: 0 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_shortening: 1 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: checksum: 4 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: no_block_cache: 0 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache: 0x556059c0d1f0 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_options: 2024-09-19T05:31:28.900 INFO:journalctl@ceph.mon.a.smithi038.stdout: capacity : 536870912 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_shard_bits : 4 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: strict_capacity_limit : 0 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_compressed: (nil) 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: persistent_cache: (nil) 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size: 4096 2024-09-19T05:31:28.901 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size_deviation: 10 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_restart_interval: 16 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_block_restart_interval: 1 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: metadata_block_size: 4096 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: partition_filters: 0 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: use_delta_encoding: 1 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: filter_policy: bloomfilter 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: whole_key_filtering: 1 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: verify_compression: 0 2024-09-19T05:31:28.902 INFO:journalctl@ceph.mon.a.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: format_version: 5 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_index_compression: 1 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_align: 0 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: max_auto_readahead_size: 262144 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: prepopulate_block_cache: 0 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: initial_auto_readahead_size: 8192 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T05:31:28.903 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression: NoCompression 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.num_levels: 7 2024-09-19T05:31:28.904 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T05:31:28.905 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:31:28.906 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T05:31:28.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T05:31:28.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:31:28.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T05:31:28.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T05:31:28.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T05:31:28.908 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T05:31:28.909 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T05:31:28.910 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T05:31:28.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.inplace_update_support: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.bloom_locality: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.max_successive_merges: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T05:31:28.912 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.ttl: 2592000 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enable_blob_files: false 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.min_blob_size: 0 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T05:31:28.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e5aa1d2f-9ff0-4da4-8a7b-37fedc8eae2d 2024-09-19T05:31:28.914 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723888603112, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723888604347, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99473, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97367, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14048, "raw_average_key_size": 49, "raw_value_size": 89695, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726723888, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e5aa1d2f-9ff0-4da4-8a7b-37fedc8eae2d", "db_session_id": "307FMT41WAZG1MNL4CJE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723888604444, "job": 1, "event": "recovery_finished"} 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556059c3ae00 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: DB pointer 0x556059d36000 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** DB Stats ** 2024-09-19T05:31:28.915 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:31:28.916 INFO:journalctl@ceph.mon.a.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T05:31:28.917 INFO:journalctl@ceph.mon.a.smithi038.stdout: L0 2/0 99.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:31:28.917 INFO:journalctl@ceph.mon.a.smithi038.stdout: Sum 2/0 99.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:31:28.917 INFO:journalctl@ceph.mon.a.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 113.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:31:28.917 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.917 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 113.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T05:31:28.918 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative compaction: 0.00 GB write, 24.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval compaction: 0.00 GB write, 24.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:31:28.919 INFO:journalctl@ceph.mon.a.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache BinnedLRUCache@0x556059c0d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: starting mon.a rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???) e1 preinit fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).mds e1 new map 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).mds e1 print_map 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: e1 2024-09-19T05:31:28.920 INFO:journalctl@ceph.mon.a.smithi038.stdout: btime 2024-09-19T05:31:23:687807+0000 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout: legacy client fscid: -1 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout: No filesystems configured 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T05:31:28.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 bash[31208]: edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: monmap epoch 1 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: last_changed 2024-09-19T05:31:20.164217+0000 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: election_strategy: 1 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: fsmap 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 ceph-mon[31227]: mgrmap e1: no daemons active 2024-09-19T05:31:28.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:28 smithi038 systemd[1]: Started Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:31:29.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.066+0000 7f00bed28640 1 Processor -- start 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.066+0000 7f00bed28640 1 -- start start 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.066+0000 7f00bed28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.066+0000 7f00bed28640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00b807be80 con 0x7f00b8107f30 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.067+0000 7f00bcac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.067+0000 7f00bcac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38470/0 (socket says 172.21.15.38:38470) 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.067+0000 7f00bcac4640 1 -- 172.21.15.38:0/510494015 learned_addr learned my addr 172.21.15.38:0/510494015 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.067+0000 7f00bcac4640 1 -- 172.21.15.38:0/510494015 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b810a860 con 0x7f00b8107f30 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.069+0000 7f00bcac4640 1 --2- 172.21.15.38:0/510494015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009820 tx=0x7f00ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c35e08fba1487 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.069+0000 7f00b77fe640 1 -- 172.21.15.38:0/510494015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac037070 con 0x7f00b8107f30 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.069+0000 7f00b77fe640 1 -- 172.21.15.38:0/510494015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00ac032900 con 0x7f00b8107f30 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.069+0000 7f00b77fe640 1 -- 172.21.15.38:0/510494015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac030850 con 0x7f00b8107f30 2024-09-19T05:31:29.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 -- 172.21.15.38:0/510494015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 msgr2=0x7f00b810a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 --2- 172.21.15.38:0/510494015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009820 tx=0x7f00ac02a270 comp rx=0 tx=0).stop 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 -- 172.21.15.38:0/510494015 shutdown_connections 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 --2- 172.21.15.38:0/510494015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b810a320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 -- 172.21.15.38:0/510494015 >> 172.21.15.38:0/510494015 conn(0x7f00b8101b90 msgr2=0x7f00b8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 -- 172.21.15.38:0/510494015 shutdown_connections 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.070+0000 7f00bed28640 1 -- 172.21.15.38:0/510494015 wait complete. 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bed28640 1 Processor -- start 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bed28640 1 -- start start 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bed28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bed28640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00b810b900 con 0x7f00b8107f30 2024-09-19T05:31:29.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bcac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bcac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38480/0 (socket says 172.21.15.38:38480) 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bcac4640 1 -- 172.21.15.38:0/2419479433 learned_addr learned my addr 172.21.15.38:0/2419479433 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bcac4640 1 -- 172.21.15.38:0/2419479433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b8107cf0 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.071+0000 7f00bcac4640 1 --2- 172.21.15.38:0/2419479433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f00ac032ce0 tx=0x7f00ac032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac042070 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00ac0308c0 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b8105ee0 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00ac037040 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b8106420 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f00ac045070 con 0x7f00b8107f30 2024-09-19T05:31:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.072+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f00ac047b70 con 0x7f00b8107f30 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.073+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b8108390 con 0x7f00b8107f30 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.075+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f00ac03c760 con 0x7f00b8107f30 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.134+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f00b8106760 con 0x7f00b8107f30 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.137+0000 7f00b5ffb640 1 -- 172.21.15.38:0/2419479433 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f00ac03c940 con 0x7f00b8107f30 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.137+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 msgr2=0x7f00b81077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.137+0000 7f00bed28640 1 --2- 172.21.15.38:0/2419479433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f00ac032ce0 tx=0x7f00ac032d10 comp rx=0 tx=0).stop 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.138+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 shutdown_connections 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.138+0000 7f00bed28640 1 --2- 172.21.15.38:0/2419479433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8107f30 0x7f00b81077b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.138+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 >> 172.21.15.38:0/2419479433 conn(0x7f00b8101b90 msgr2=0x7f00b81026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.138+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 shutdown_connections 2024-09-19T05:31:29.657 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:29.138+0000 7f00bed28640 1 -- 172.21.15.38:0/2419479433 wait complete. 2024-09-19T05:31:29.658 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T05:31:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:31:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-09-19T05:31:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T05:31:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T05:31:29.950 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x 2024-09-19T05:31:29.951 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T05:31:30.107 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968.target.wants/ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-44ff0e58-7648-11ef-bceb-c7b262605968@.service. 2024-09-19T05:31:30.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:30 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2419479433' entity='client.admin' 2024-09-19T05:31:30.395 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 podman[31437]: 2024-09-19 05:31:30.339030326 +0000 UTC m=+0.117458636 container create f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T05:31:30.643 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:31:30.643 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-19T05:31:30.652 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:31:30.652 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:31:30.652 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-19T05:31:30.652 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T05:31:30.663 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:31:30.663 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-09-19T05:31:30.672 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-09-19T05:31:30.745 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 podman[31437]: 2024-09-19 05:31:30.541397953 +0000 UTC m=+0.319826280 container init f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T05:31:30.745 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 podman[31437]: 2024-09-19 05:31:30.544173143 +0000 UTC m=+0.322601454 container start f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T05:31:30.745 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 bash[31437]: f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 2024-09-19T05:31:30.745 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 systemd[1]: Started Ceph mgr.x for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:31:30.746 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:30.677+0000 7f5ab14bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:31:30.746 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:30.743+0000 7f5ab14bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:31:31.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:30.904+0000 7f5ab14bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:31:31.029 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:30.977+0000 7f5ab14bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:31:31.295 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/626134031' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:31:31.296 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:31.137+0000 7f5ab14bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:31:31.296 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:31.293+0000 7f5ab14bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:31:31.888 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:31.627+0000 7f5ab14bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:31:31.888 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:31.691+0000 7f5ab14bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:31:31.888 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:31 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:31.886+0000 7f5ab14bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:31:32.249 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.250 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.251 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:31:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:31:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:31:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:31:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:31:32.252 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:31:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:31:23:687807+0000", 2024-09-19T05:31:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:31:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:31:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T05:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:31:32.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:31:32.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:31:23.688683+0000", 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:31:32.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.108+0000 7f487ab3d640 1 Processor -- start 2024-09-19T05:31:32.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.108+0000 7f487ab3d640 1 -- start start 2024-09-19T05:31:32.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.108+0000 7f487ab3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:32.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.108+0000 7f487ab3d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4874105fc0 con 0x7f4874107a80 2024-09-19T05:31:32.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.109+0000 7f48788d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:32.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.109+0000 7f48788d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38520/0 (socket says 172.21.15.38:38520) 2024-09-19T05:31:32.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.109+0000 7f48788d9640 1 -- 172.21.15.38:0/762022360 learned_addr learned my addr 172.21.15.38:0/762022360 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:32.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.109+0000 7f48788d9640 1 -- 172.21.15.38:0/762022360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4874106140 con 0x7f4874107a80 2024-09-19T05:31:32.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.110+0000 7f48788d9640 1 --2- 172.21.15.38:0/762022360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f485c009820 tx=0x7f485c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8470c5ebfe14cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:32.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.110+0000 7f48737fe640 1 -- 172.21.15.38:0/762022360 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f485c037070 con 0x7f4874107a80 2024-09-19T05:31:32.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.110+0000 7f48737fe640 1 -- 172.21.15.38:0/762022360 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f485c032900 con 0x7f4874107a80 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.110+0000 7f48737fe640 1 -- 172.21.15.38:0/762022360 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f485c030850 con 0x7f4874107a80 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 -- 172.21.15.38:0/762022360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 msgr2=0x7f4874107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 --2- 172.21.15.38:0/762022360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f485c009820 tx=0x7f485c02a270 comp rx=0 tx=0).stop 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 -- 172.21.15.38:0/762022360 shutdown_connections 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 --2- 172.21.15.38:0/762022360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f4874107e80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 -- 172.21.15.38:0/762022360 >> 172.21.15.38:0/762022360 conn(0x7f4874101af0 msgr2=0x7f4874103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 -- 172.21.15.38:0/762022360 shutdown_connections 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 -- 172.21.15.38:0/762022360 wait complete. 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.111+0000 7f487ab3d640 1 Processor -- start 2024-09-19T05:31:32.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f487ab3d640 1 -- start start 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f487ab3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f487ab3d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f487410a950 con 0x7f4874107a80 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f48788d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f48788d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38526/0 (socket says 172.21.15.38:38526) 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f48788d9640 1 -- 172.21.15.38:0/626134031 learned_addr learned my addr 172.21.15.38:0/626134031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f48788d9640 1 -- 172.21.15.38:0/626134031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487419bb60 con 0x7f4874107a80 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f48788d9640 1 --2- 172.21.15.38:0/626134031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f485c009940 tx=0x7f485c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:32.261 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f485c042070 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.112+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f485c0308c0 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.113+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487419bdf0 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.113+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f485c037040 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.113+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487419eae0 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.113+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f485c045070 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.113+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f485c047b70 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.114+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4874107f00 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.116+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f485c0327b0 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.178+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f487402d940 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.179+0000 7f4871ffb640 1 -- 172.21.15.38:0/626134031 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f485c03c760 con 0x7f4874107a80 2024-09-19T05:31:32.262 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 msgr2=0x7f487419b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 --2- 172.21.15.38:0/626134031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f485c009940 tx=0x7f485c032ce0 comp rx=0 tx=0).stop 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 shutdown_connections 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 --2- 172.21.15.38:0/626134031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4874107a80 0x7f487419b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 >> 172.21.15.38:0/626134031 conn(0x7f4874101af0 msgr2=0x7f4874102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 shutdown_connections 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:31.180+0000 7f487ab3d640 1 -- 172.21.15.38:0/626134031 wait complete. 2024-09-19T05:31:32.263 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-09-19T05:31:33.144 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:32.817+0000 7f5ab14bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:31:33.144 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:32.887+0000 7f5ab14bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:31:33.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:33 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:33.666+0000 7f5ab14bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:31:33.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:33 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:33.787+0000 7f5ab14bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:31:34.233 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:33 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:33.937+0000 7f5ab14bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:31:34.233 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.002+0000 7f5ab14bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:31:34.233 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.159+0000 7f5ab14bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:31:34.233 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.230+0000 7f5ab14bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:31:34.483 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.297+0000 7f5ab14bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:31:34.484 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.465+0000 7f5ab14bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:31:34.738 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.535+0000 7f5ab14bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:31:34.738 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.606+0000 7f5ab14bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:31:34.738 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.697+0000 7f5ab14bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:31:35.147 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3784265977' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:31:35.147 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.766+0000 7f5ab14bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:31:35.147 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:34.835+0000 7f5ab14bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:31:35.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:35.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:31:35.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:31:35.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:31:35.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:35.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:31:35.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:35.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:31:23:687807+0000", 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:31:23.688683+0000", 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.655+0000 7f236cac0640 1 Processor -- start 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236cac0640 1 -- start start 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236cac0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236cac0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2368075900 con 0x7f2368107f90 2024-09-19T05:31:35.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38530/0 (socket says 172.21.15.38:38530) 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236659c640 1 -- 172.21.15.38:0/2319950957 learned_addr learned my addr 172.21.15.38:0/2319950957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.656+0000 7f236659c640 1 -- 172.21.15.38:0/2319950957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236810a8c0 con 0x7f2368107f90 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.657+0000 7f236659c640 1 --2- 172.21.15.38:0/2319950957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2358009820 tx=0x7f235802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b364c9186e0e3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.657+0000 7f2365d9b640 1 -- 172.21.15.38:0/2319950957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2358037070 con 0x7f2368107f90 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.657+0000 7f2365d9b640 1 -- 172.21.15.38:0/2319950957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2358032900 con 0x7f2368107f90 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.657+0000 7f2365d9b640 1 -- 172.21.15.38:0/2319950957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2358030850 con 0x7f2368107f90 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 -- 172.21.15.38:0/2319950957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 msgr2=0x7f236810a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 --2- 172.21.15.38:0/2319950957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2358009820 tx=0x7f235802a270 comp rx=0 tx=0).stop 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 -- 172.21.15.38:0/2319950957 shutdown_connections 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 --2- 172.21.15.38:0/2319950957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236810a380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:35.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 -- 172.21.15.38:0/2319950957 >> 172.21.15.38:0/2319950957 conn(0x7f2368101af0 msgr2=0x7f2368103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 -- 172.21.15.38:0/2319950957 shutdown_connections 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.658+0000 7f236cac0640 1 -- 172.21.15.38:0/2319950957 wait complete. 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236cac0640 1 Processor -- start 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236cac0640 1 -- start start 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236cac0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236cac0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f236810b9c0 con 0x7f2368107f90 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38540/0 (socket says 172.21.15.38:38540) 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236659c640 1 -- 172.21.15.38:0/3784265977 learned_addr learned my addr 172.21.15.38:0/3784265977 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236659c640 1 -- 172.21.15.38:0/3784265977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236819bae0 con 0x7f2368107f90 2024-09-19T05:31:35.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.659+0000 7f236659c640 1 --2- 172.21.15.38:0/3784265977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23580097f0 tx=0x7f2358030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2358037040 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236819bd70 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2358040070 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236819ea60 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235803a390 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f235803a670 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.660+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f23580459b0 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.661+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23681083f0 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.663+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2358045460 con 0x7f2368107f90 2024-09-19T05:31:35.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.725+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f236819ef20 con 0x7f2368107f90 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.726+0000 7f235f7fe640 1 -- 172.21.15.38:0/3784265977 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f235803b430 con 0x7f2368107f90 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 msgr2=0x7f236819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 --2- 172.21.15.38:0/3784265977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23580097f0 tx=0x7f2358030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 shutdown_connections 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 --2- 172.21.15.38:0/3784265977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2368107f90 0x7f236819b5a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 >> 172.21.15.38:0/3784265977 conn(0x7f2368101af0 msgr2=0x7f23681026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 shutdown_connections 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:34.727+0000 7f236cac0640 1 -- 172.21.15.38:0/3784265977 wait complete. 2024-09-19T05:31:35.396 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:35.681+0000 7f5ab14bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: Activating manager daemon x 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: mgrmap e2: x(active, starting, since 0.00325801s) 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:31:36.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: Manager daemon x is now available 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' 2024-09-19T05:31:36.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:35 smithi038 ceph-mon[31227]: from='mgr.14100 172.21.15.38:0/2756658993' entity='mgr.x' 2024-09-19T05:31:37.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:37 smithi038 ceph-mon[31227]: mgrmap e3: x(active, since 1.00903s) 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T05:31:38.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T05:31:38.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T05:31:38.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:38.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T05:31:38.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T05:31:38.640 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T05:31:38.648 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T05:31:38.648 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T05:31:38.649 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T05:31:23:687807+0000", 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T05:31:38.650 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T05:31:38.651 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T05:31:23.688683+0000", 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.756+0000 7f97100cb640 1 Processor -- start 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.756+0000 7f97100cb640 1 -- start start 2024-09-19T05:31:38.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f97100cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f97100cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9708105fc0 con 0x7f9708107a80 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f970de67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f970de67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32806/0 (socket says 172.21.15.38:32806) 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f970de67640 1 -- 172.21.15.38:0/13890246 learned_addr learned my addr 172.21.15.38:0/13890246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.757+0000 7f970de67640 1 -- 172.21.15.38:0/13890246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708106140 con 0x7f9708107a80 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.758+0000 7f970de67640 1 --2- 172.21.15.38:0/13890246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009e80 tx=0x7f96f802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db02db23c50f3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.758+0000 7f970ce65640 1 -- 172.21.15.38:0/13890246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f8036070 con 0x7f9708107a80 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.758+0000 7f970ce65640 1 -- 172.21.15.38:0/13890246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96f80318c0 con 0x7f9708107a80 2024-09-19T05:31:38.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.758+0000 7f970ce65640 1 -- 172.21.15.38:0/13890246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f8007ab0 con 0x7f9708107a80 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 -- 172.21.15.38:0/13890246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 msgr2=0x7f9708107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 --2- 172.21.15.38:0/13890246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009e80 tx=0x7f96f802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 -- 172.21.15.38:0/13890246 shutdown_connections 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 --2- 172.21.15.38:0/13890246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f9708107e80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 -- 172.21.15.38:0/13890246 >> 172.21.15.38:0/13890246 conn(0x7f9708101af0 msgr2=0x7f9708103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 -- 172.21.15.38:0/13890246 shutdown_connections 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.759+0000 7f97100cb640 1 -- 172.21.15.38:0/13890246 wait complete. 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f97100cb640 1 Processor -- start 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f97100cb640 1 -- start start 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f97100cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f97100cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f970810a950 con 0x7f9708107a80 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f970de67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:38.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f970de67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32810/0 (socket says 172.21.15.38:32810) 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f970de67640 1 -- 172.21.15.38:0/2728520437 learned_addr learned my addr 172.21.15.38:0/2728520437 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.760+0000 7f970de67640 1 -- 172.21.15.38:0/2728520437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708191bf0 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f970de67640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96f802ad90 tx=0x7f96f8031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f8036050 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96f8042070 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970818fde0 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96f803d7e0 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.761+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708190320 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.762+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f96f803dae0 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.762+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9708107f00 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.762+0000 7f9706ffd640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 0x7f96f003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.762+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96f8007e00 con 0x7f9708107a80 2024-09-19T05:31:38.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.763+0000 7f970d666640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 0x7f96f003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.763+0000 7f970d666640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 0x7f96f003b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0098c0 tx=0x7f96fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:37.768+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96f8039de0 con 0x7f9708107a80 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.013+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9708195390 con 0x7f9708107a80 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.014+0000 7f9706ffd640 1 -- 172.21.15.38:0/2728520437 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f96f803c9c0 con 0x7f9708107a80 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 msgr2=0x7f96f003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 0x7f96f003b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0098c0 tx=0x7f96fc006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 msgr2=0x7f97081916b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96f802ad90 tx=0x7f96f8031ca0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 shutdown_connections 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f96f0039160 0x7f96f003b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 --2- 172.21.15.38:0/2728520437 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9708107a80 0x7f97081916b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 >> 172.21.15.38:0/2728520437 conn(0x7f9708101af0 msgr2=0x7f9708102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 shutdown_connections 2024-09-19T05:31:38.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:38.016+0000 7f97100cb640 1 -- 172.21.15.38:0/2728520437 wait complete. 2024-09-19T05:31:38.657 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-09-19T05:31:38.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:38 smithi038 ceph-mon[31227]: mgrmap e4: x(active, since 2s) 2024-09-19T05:31:38.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:38 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2728520437' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T05:31:39.865 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2622f02640 1 Processor -- start 2024-09-19T05:31:39.866 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2622f02640 1 -- start start 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2622f02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2622f02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f261c105fc0 con 0x7f261c107a80 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2620c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2620c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32822/0 (socket says 172.21.15.38:32822) 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.055+0000 7f2620c9e640 1 -- 172.21.15.38:0/1015025369 learned_addr learned my addr 172.21.15.38:0/1015025369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.056+0000 7f2620c9e640 1 -- 172.21.15.38:0/1015025369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c106140 con 0x7f261c107a80 2024-09-19T05:31:39.867 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.056+0000 7f2620c9e640 1 --2- 172.21.15.38:0/1015025369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2610009820 tx=0x7f261002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d779e049a725774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f261b7fe640 1 -- 172.21.15.38:0/1015025369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2610037070 con 0x7f261c107a80 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f261b7fe640 1 -- 172.21.15.38:0/1015025369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2610032900 con 0x7f261c107a80 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f261b7fe640 1 -- 172.21.15.38:0/1015025369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2610030850 con 0x7f261c107a80 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 -- 172.21.15.38:0/1015025369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 msgr2=0x7f261c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 --2- 172.21.15.38:0/1015025369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2610009820 tx=0x7f261002a270 comp rx=0 tx=0).stop 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 -- 172.21.15.38:0/1015025369 shutdown_connections 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 --2- 172.21.15.38:0/1015025369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c107e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 -- 172.21.15.38:0/1015025369 >> 172.21.15.38:0/1015025369 conn(0x7f261c101af0 msgr2=0x7f261c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 -- 172.21.15.38:0/1015025369 shutdown_connections 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.057+0000 7f2622f02640 1 -- 172.21.15.38:0/1015025369 wait complete. 2024-09-19T05:31:39.868 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.058+0000 7f2622f02640 1 Processor -- start 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.058+0000 7f2622f02640 1 -- start start 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.058+0000 7f2622f02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.058+0000 7f2622f02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10a950 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2620c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2620c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32836/0 (socket says 172.21.15.38:32836) 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2620c9e640 1 -- 172.21.15.38:0/2579881377 learned_addr learned my addr 172.21.15.38:0/2579881377 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2620c9e640 1 -- 172.21.15.38:0/2579881377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c19e300 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2620c9e640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2610009940 tx=0x7f2610032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2610042070 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26100308c0 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c19e4d0 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2610037040 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.059+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c19e8f0 con 0x7f261c107a80 2024-09-19T05:31:39.869 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.060+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2610045070 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.061+0000 7f2619ffb640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 0x7f260c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.061+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f261006c1d0 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.061+0000 7f261bfff640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 0x7f260c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.061+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c107f00 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.061+0000 7f261bfff640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 0x7f260c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26080098c0 tx=0x7f2608006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.066+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f261002ed90 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.240+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f261c19eb20 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.241+0000 7f2619ffb640 1 -- 172.21.15.38:0/2579881377 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f261003c760 con 0x7f261c107a80 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 msgr2=0x7f260c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:39.870 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 0x7f260c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26080098c0 tx=0x7f2608006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 msgr2=0x7f261c075460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2610009940 tx=0x7f2610032ce0 comp rx=0 tx=0).stop 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 shutdown_connections 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f260c0391b0 0x7f260c03b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 --2- 172.21.15.38:0/2579881377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c107a80 0x7f261c075460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 >> 172.21.15.38:0/2579881377 conn(0x7f261c101af0 msgr2=0x7f261c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 shutdown_connections 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:39.243+0000 7f2622f02640 1 -- 172.21.15.38:0/2579881377 wait complete. 2024-09-19T05:31:39.871 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-09-19T05:31:39.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:39 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2579881377' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T05:31:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:40 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/722634876' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T05:31:40.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ignoring --setuser ceph since I am not root 2024-09-19T05:31:40.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ignoring --setgroup ceph since I am not root 2024-09-19T05:31:40.895 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:40.847+0000 7f4b3e419040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:31:41.170 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:40.907+0000 7f4b3e419040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:31:41.170 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.060+0000 7f4b3e419040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:31:41.170 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.132+0000 7f4b3e419040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:31:41.438 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.288+0000 7f4b3e419040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:31:41.438 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.436+0000 7f4b3e419040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:31:41.481 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29edf12640 1 Processor -- start 2024-09-19T05:31:41.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29edf12640 1 -- start start 2024-09-19T05:31:41.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29edf12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:41.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29edf12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29e807c7d0 con 0x7f29e807dde0 2024-09-19T05:31:41.487 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32852/0 (socket says 172.21.15.38:32852) 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.409+0000 7f29e77fe640 1 -- 172.21.15.38:0/1024406086 learned_addr learned my addr 172.21.15.38:0/1024406086 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.410+0000 7f29e77fe640 1 -- 172.21.15.38:0/1024406086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e807c950 con 0x7f29e807dde0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.410+0000 7f29e77fe640 1 --2- 172.21.15.38:0/1024406086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009820 tx=0x7f29d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a5466f40d1c089 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29e67fc640 1 -- 172.21.15.38:0/1024406086 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8037070 con 0x7f29e807dde0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29e67fc640 1 -- 172.21.15.38:0/1024406086 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29d8032900 con 0x7f29e807dde0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29e67fc640 1 -- 172.21.15.38:0/1024406086 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8030850 con 0x7f29e807dde0 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 -- 172.21.15.38:0/1024406086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 msgr2=0x7f29e807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 --2- 172.21.15.38:0/1024406086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009820 tx=0x7f29d802a270 comp rx=0 tx=0).stop 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 -- 172.21.15.38:0/1024406086 shutdown_connections 2024-09-19T05:31:41.488 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 --2- 172.21.15.38:0/1024406086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e807c290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 -- 172.21.15.38:0/1024406086 >> 172.21.15.38:0/1024406086 conn(0x7f29e8102ce0 msgr2=0x7f29e8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 -- 172.21.15.38:0/1024406086 shutdown_connections 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.411+0000 7f29edf12640 1 -- 172.21.15.38:0/1024406086 wait complete. 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29edf12640 1 Processor -- start 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29edf12640 1 -- start start 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29edf12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29edf12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29e810db60 con 0x7f29e807dde0 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:41.489 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32860/0 (socket says 172.21.15.38:32860) 2024-09-19T05:31:41.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.412+0000 7f29e77fe640 1 -- 172.21.15.38:0/722634876 learned_addr learned my addr 172.21.15.38:0/722634876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:41.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29e77fe640 1 -- 172.21.15.38:0/722634876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e819dbd0 con 0x7f29e807dde0 2024-09-19T05:31:41.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29e77fe640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f29d80097f0 tx=0x7f29d8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:41.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8037040 con 0x7f29e807dde0 2024-09-19T05:31:41.490 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29d8040070 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e819de60 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d803a390 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.413+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e81a0b50 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.414+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f29d803a5e0 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.414+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e807c310 con 0x7f29e807dde0 2024-09-19T05:31:41.491 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.414+0000 7f29e4ff9640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 0x7f29d003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.414+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29d806c500 con 0x7f29e807dde0 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.414+0000 7f29e6ffd640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 0x7f29d003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.415+0000 7f29e6ffd640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 0x7f29d003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0098c0 tx=0x7f29dc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.419+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29d803a980 con 0x7f29e807dde0 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.642+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f29e819e2e0 con 0x7f29e807dde0 2024-09-19T05:31:41.492 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.700+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f29d806c260 con 0x7f29e807dde0 2024-09-19T05:31:41.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.702+0000 7f29e4ff9640 1 -- 172.21.15.38:0/722634876 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f29d8045460 con 0x7f29e807dde0 2024-09-19T05:31:41.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 msgr2=0x7f29d003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:41.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 0x7f29d003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0098c0 tx=0x7f29dc006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:41.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 msgr2=0x7f29e819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:41.493 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f29d80097f0 tx=0x7f29d8030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:41.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 shutdown_connections 2024-09-19T05:31:41.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f29d00391b0 0x7f29d003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:41.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 --2- 172.21.15.38:0/722634876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29e807dde0 0x7f29e819d690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:41.494 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.703+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 >> 172.21.15.38:0/722634876 conn(0x7f29e8102ce0 msgr2=0x7f29e8103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:41.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.704+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 shutdown_connections 2024-09-19T05:31:41.495 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:40.704+0000 7f29edf12640 1 -- 172.21.15.38:0/722634876 wait complete. 2024-09-19T05:31:42.037 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/722634876' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T05:31:42.037 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-mon[31227]: mgrmap e5: x(active, since 5s) 2024-09-19T05:31:42.037 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.772+0000 7f4b3e419040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:31:42.037 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:41.836+0000 7f4b3e419040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:31:42.037 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:42.033+0000 7f4b3e419040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:31:42.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.893+0000 7f78502bf640 1 Processor -- start 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.893+0000 7f78502bf640 1 -- start start 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f78502bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f78502bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7848075900 con 0x7f7848105d80 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f784e05b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f784e05b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32902/0 (socket says 172.21.15.38:32902) 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f784e05b640 1 -- 172.21.15.38:0/2975336961 learned_addr learned my addr 172.21.15.38:0/2975336961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.894+0000 7f784e05b640 1 -- 172.21.15.38:0/2975336961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481066c0 con 0x7f7848105d80 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.895+0000 7f784e05b640 1 --2- 172.21.15.38:0/2975336961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7844009820 tx=0x7f784402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ea1e030d09e7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.895+0000 7f784d059640 1 -- 172.21.15.38:0/2975336961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844037070 con 0x7f7848105d80 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.895+0000 7f784d059640 1 -- 172.21.15.38:0/2975336961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7844032900 con 0x7f7848105d80 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.895+0000 7f784d059640 1 -- 172.21.15.38:0/2975336961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844030850 con 0x7f7848105d80 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- 172.21.15.38:0/2975336961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 msgr2=0x7f7848106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:42.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 --2- 172.21.15.38:0/2975336961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7844009820 tx=0x7f784402a270 comp rx=0 tx=0).stop 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- 172.21.15.38:0/2975336961 shutdown_connections 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 --2- 172.21.15.38:0/2975336961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f7848106180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- 172.21.15.38:0/2975336961 >> 172.21.15.38:0/2975336961 conn(0x7f7848101af0 msgr2=0x7f7848103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- 172.21.15.38:0/2975336961 shutdown_connections 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- 172.21.15.38:0/2975336961 wait complete. 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 Processor -- start 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.896+0000 7f78502bf640 1 -- start start 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f78502bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f78502bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f78481077c0 con 0x7f7848105d80 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f784e05b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f784e05b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32912/0 (socket says 172.21.15.38:32912) 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f784e05b640 1 -- 172.21.15.38:0/2169717491 learned_addr learned my addr 172.21.15.38:0/2169717491 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:42.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f784e05b640 1 -- 172.21.15.38:0/2169717491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784819dce0 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f784e05b640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78440097f0 tx=0x7f7844030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844037040 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784819df70 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7844040070 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f784403a390 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.897+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a0c60 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.898+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f784403a630 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.898+0000 7f783f7fe640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 0x7f783403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.898+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f784406d4e0 con 0x7f7848105d80 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.899+0000 7f784d85a640 1 -- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 msgr2=0x7f783403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.899+0000 7f784d85a640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 0x7f783403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:31:42.681 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.899+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7848106180 con 0x7f7848105d80 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:41.903+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f784404a200 con 0x7f7848105d80 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.100+0000 7f784d85a640 1 -- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 msgr2=0x7f783403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.100+0000 7f784d85a640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 0x7f783403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.132+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7848196c10 con 0x7f7848105d80 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.132+0000 7f783f7fe640 1 -- 172.21.15.38:0/2169717491 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f784402ee80 con 0x7f7848105d80 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 msgr2=0x7f783403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 0x7f783403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 msgr2=0x7f784819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78440097f0 tx=0x7f7844030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 shutdown_connections 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f78340391b0 0x7f783403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 --2- 172.21.15.38:0/2169717491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7848105d80 0x7f784819d7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 >> 172.21.15.38:0/2169717491 conn(0x7f7848101af0 msgr2=0x7f7848191aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:42.682 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 shutdown_connections 2024-09-19T05:31:42.683 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:42.135+0000 7f78502bf640 1 -- 172.21.15.38:0/2169717491 wait complete. 2024-09-19T05:31:42.683 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-19T05:31:42.683 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-09-19T05:31:42.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:42 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2169717491' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T05:31:43.342 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:42.970+0000 7f4b3e419040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:31:43.342 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:43.041+0000 7f4b3e419040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:31:44.059 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:43.788+0000 7f4b3e419040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:31:44.060 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:43.909+0000 7f4b3e419040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:31:44.350 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.058+0000 7f4b3e419040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:31:44.350 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.121+0000 7f4b3e419040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:31:44.350 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.277+0000 7f4b3e419040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:31:44.350 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.348+0000 7f4b3e419040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:31:44.644 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.411+0000 7f4b3e419040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:31:44.644 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.576+0000 7f4b3e419040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:31:44.925 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.644+0000 7f4b3e419040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:31:44.925 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.713+0000 7f4b3e419040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:31:44.925 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.800+0000 7f4b3e419040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:31:44.925 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.862+0000 7f4b3e419040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:31:44.925 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:44.923+0000 7f4b3e419040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:45.750+0000 7f4b3e419040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: Active manager daemon x restarted 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: Activating manager daemon x 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: mgrmap e6: x(active, starting, since 0.00793294s) 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:31:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: Manager daemon x is now available 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:31:45.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:45 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: Found migration_current of "None". Setting to last migration. 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:47.051 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:46 smithi038 ceph-mon[31227]: mgrmap e7: x(active, since 1.00566s) 2024-09-19T05:31:47.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:31:47.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T05:31:47.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f46963cd640 1 Processor -- start 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f46963cd640 1 -- start start 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f46963cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f46963cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46900758a0 con 0x7f4690107ff0 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f468ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f468ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32918/0 (socket says 172.21.15.38:32918) 2024-09-19T05:31:47.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.134+0000 7f468ffff640 1 -- 172.21.15.38:0/4281889348 learned_addr learned my addr 172.21.15.38:0/4281889348 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.135+0000 7f468ffff640 1 -- 172.21.15.38:0/4281889348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469010a920 con 0x7f4690107ff0 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.135+0000 7f468ffff640 1 --2- 172.21.15.38:0/4281889348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4680009820 tx=0x7f468002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa122f3d9771f2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.135+0000 7f468effd640 1 -- 172.21.15.38:0/4281889348 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4680037070 con 0x7f4690107ff0 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f468effd640 1 -- 172.21.15.38:0/4281889348 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4680032900 con 0x7f4690107ff0 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f468effd640 1 -- 172.21.15.38:0/4281889348 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4680030850 con 0x7f4690107ff0 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 -- 172.21.15.38:0/4281889348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 msgr2=0x7f469010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:47.255 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 --2- 172.21.15.38:0/4281889348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4680009820 tx=0x7f468002a270 comp rx=0 tx=0).stop 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 -- 172.21.15.38:0/4281889348 shutdown_connections 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 --2- 172.21.15.38:0/4281889348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469010a3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 -- 172.21.15.38:0/4281889348 >> 172.21.15.38:0/4281889348 conn(0x7f4690101af0 msgr2=0x7f4690103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 -- 172.21.15.38:0/4281889348 shutdown_connections 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.136+0000 7f46963cd640 1 -- 172.21.15.38:0/4281889348 wait complete. 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f46963cd640 1 Processor -- start 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f46963cd640 1 -- start start 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f46963cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f46963cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f469010b9c0 con 0x7f4690107ff0 2024-09-19T05:31:47.256 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f468ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f468ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32920/0 (socket says 172.21.15.38:32920) 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.137+0000 7f468ffff640 1 -- 172.21.15.38:0/2956224843 learned_addr learned my addr 172.21.15.38:0/2956224843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f468ffff640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019dd00 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f468ffff640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f46800097f0 tx=0x7f4680030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4680037040 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4680040070 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019df90 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f468003a390 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.138+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a0c80 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f468003a5e0 con 0x7f4690107ff0 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f468d7fa640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4690108450 con 0x7f4678039200 2024-09-19T05:31:47.257 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f468006c400 con 0x7f4690107ff0 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f468f7fe640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 msgr2=0x7f467803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.139+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.340+0000 7f468f7fe640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 msgr2=0x7f467803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.340+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.741+0000 7f468f7fe640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 msgr2=0x7f467803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:43.741+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:44.542+0000 7f468f7fe640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 msgr2=0x7f467803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4202404879 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:44.542+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:45.758+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f468002fd60 con 0x7f4690107ff0 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:45.758+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 msgr2=0x7f467803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:45.758+0000 7f468d7fa640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.755+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f468003a8d0 con 0x7f4690107ff0 2024-09-19T05:31:47.258 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.755+0000 7f468d7fa640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 0x7f467803ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.755+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f468003de70 con 0x7f467803c940 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.755+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 0x7f467803ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.756+0000 7f468f7fe640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 0x7f467803ed30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4684003980 tx=0x7f46840092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.757+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f468003de70 con 0x7f467803c940 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4690102d80 con 0x7f467803c940 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f468d7fa640 1 -- 172.21.15.38:0/2956224843 <== mgr.14118 v2:172.21.15.38:6800/3555821050 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4690102d80 con 0x7f467803c940 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 msgr2=0x7f467803ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 0x7f467803ed30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4684003980 tx=0x7f46840092b0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 msgr2=0x7f469019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f46800097f0 tx=0x7f4680030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 shutdown_connections 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f467803c940 0x7f467803ed30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.259 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:6800/4202404879,v1:172.21.15.38:6801/4202404879] conn(0x7f4678039200 0x7f467803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 --2- 172.21.15.38:0/2956224843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4690107ff0 0x7f469019d7c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 >> 172.21.15.38:0/2956224843 conn(0x7f4690101af0 msgr2=0x7f4690102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 shutdown_connections 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:46.764+0000 7f46963cd640 1 -- 172.21.15.38:0/2956224843 wait complete. 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-09-19T05:31:47.260 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-09-19T05:31:48.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:47 smithi038 ceph-mon[31227]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T05:31:48.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:47 smithi038 ceph-mon[31227]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.682+0000 7fd3fae10640 1 Processor -- start 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.682+0000 7fd3fae10640 1 -- start start 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.682+0000 7fd3fae10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.682+0000 7fd3fae10640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410a840 con 0x7fd3f4107f10 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.683+0000 7fd3f8bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.683+0000 7fd3f8bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53206/0 (socket says 172.21.15.38:53206) 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.683+0000 7fd3f8bac640 1 -- 172.21.15.38:0/4088872000 learned_addr learned my addr 172.21.15.38:0/4088872000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.683+0000 7fd3f8bac640 1 -- 172.21.15.38:0/4088872000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f410a9c0 con 0x7fd3f4107f10 2024-09-19T05:31:48.402 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3f8bac640 1 --2- 172.21.15.38:0/4088872000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009820 tx=0x7fd3e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eacea89ba4096 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3f37fe640 1 -- 172.21.15.38:0/4088872000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8037070 con 0x7fd3f4107f10 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3f37fe640 1 -- 172.21.15.38:0/4088872000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd3e8032900 con 0x7fd3f4107f10 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3f37fe640 1 -- 172.21.15.38:0/4088872000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8030850 con 0x7fd3f4107f10 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3fae10640 1 -- 172.21.15.38:0/4088872000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 msgr2=0x7fd3f410a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3fae10640 1 --2- 172.21.15.38:0/4088872000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009820 tx=0x7fd3e802a270 comp rx=0 tx=0).stop 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3fae10640 1 -- 172.21.15.38:0/4088872000 shutdown_connections 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3fae10640 1 --2- 172.21.15.38:0/4088872000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f410a300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.684+0000 7fd3fae10640 1 -- 172.21.15.38:0/4088872000 >> 172.21.15.38:0/4088872000 conn(0x7fd3f4101af0 msgr2=0x7fd3f4103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.685+0000 7fd3fae10640 1 -- 172.21.15.38:0/4088872000 shutdown_connections 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.685+0000 7fd3fae10640 1 -- 172.21.15.38:0/4088872000 wait complete. 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.685+0000 7fd3fae10640 1 Processor -- start 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.685+0000 7fd3fae10640 1 -- start start 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3fae10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:48.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3fae10640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410b9a0 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f8bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f8bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53208/0 (socket says 172.21.15.38:53208) 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f8bac640 1 -- 172.21.15.38:0/102425065 learned_addr learned my addr 172.21.15.38:0/102425065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f8bac640 1 -- 172.21.15.38:0/102425065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f419baf0 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f8bac640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8032ce0 tx=0x7fd3e8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8042070 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd3e80308c0 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f419bd80 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.686+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8037040 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.687+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f419ea70 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.687+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fd3e8045070 con 0x7fd3f4107f10 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.688+0000 7fd3f1ffb640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 0x7fd3e403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:48.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.688+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd3e806c3a0 con 0x7fd3f4107f10 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.688+0000 7fd3f3fff640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 0x7fd3e403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.688+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f4108370 con 0x7fd3f4107f10 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.688+0000 7fd3f3fff640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 0x7fd3e403b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098c0 tx=0x7fd3e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.693+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3e803add0 con 0x7fd3f4107f10 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.823+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd3e803c780 con 0x7fd3f4107f10 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.896+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd3f4102dc0 con 0x7fd3e40390f0 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.901+0000 7fd3f1ffb640 1 -- 172.21.15.38:0/102425065 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3f4102dc0 con 0x7fd3e40390f0 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 msgr2=0x7fd3e403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 0x7fd3e403b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00098c0 tx=0x7fd3e0006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 msgr2=0x7fd3f419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8032ce0 tx=0x7fd3e8032d10 comp rx=0 tx=0).stop 2024-09-19T05:31:48.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 shutdown_connections 2024-09-19T05:31:48.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fd3e40390f0 0x7fd3e403b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:48.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 --2- 172.21.15.38:0/102425065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3f4107f10 0x7fd3f419b5b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:48.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 >> 172.21.15.38:0/102425065 conn(0x7fd3f4101af0 msgr2=0x7fd3f41026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:48.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 shutdown_connections 2024-09-19T05:31:48.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:47.903+0000 7fd3fae10640 1 -- 172.21.15.38:0/102425065 wait complete. 2024-09-19T05:31:49.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:48 smithi038 ceph-mon[31227]: mgrmap e8: x(active, since 2s) 2024-09-19T05:31:49.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:48 smithi038 ceph-mon[31227]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:49.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:48 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:49.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:48 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:31:49.593 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:49.275+0000 7f4aa5733640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:31:49] ENGINE Error in HTTPServer.serve 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: Traceback (most recent call last): 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._connections.run(self.expiration_interval) 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._run(expiration_interval) 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:31:49.594 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: s = self.context.wrap_socket( 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:31:49.595 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: return self.sslsocket_class._create( 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self.do_handshake() 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._sslobj.do_handshake() 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:31:49.596 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:31:49.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T05:31:49.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.846+0000 7fe8af294640 1 Processor -- start 2024-09-19T05:31:49.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.846+0000 7fe8af294640 1 -- start start 2024-09-19T05:31:49.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8af294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:49.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8af294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8075900 con 0x7fe8a8105d80 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8ad030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8ad030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53224/0 (socket says 172.21.15.38:53224) 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8ad030640 1 -- 172.21.15.38:0/1813407328 learned_addr learned my addr 172.21.15.38:0/1813407328 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.847+0000 7fe8ad030640 1 -- 172.21.15.38:0/1813407328 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a81066c0 con 0x7fe8a8105d80 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.848+0000 7fe8ad030640 1 --2- 172.21.15.38:0/1813407328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009820 tx=0x7fe8a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61864c2dea05577 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.848+0000 7fe89bfff640 1 -- 172.21.15.38:0/1813407328 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8a4037070 con 0x7fe8a8105d80 2024-09-19T05:31:49.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.848+0000 7fe89bfff640 1 -- 172.21.15.38:0/1813407328 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8a4032900 con 0x7fe8a8105d80 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.848+0000 7fe89bfff640 1 -- 172.21.15.38:0/1813407328 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8a4030850 con 0x7fe8a8105d80 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- 172.21.15.38:0/1813407328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 msgr2=0x7fe8a8106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 --2- 172.21.15.38:0/1813407328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009820 tx=0x7fe8a402a270 comp rx=0 tx=0).stop 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- 172.21.15.38:0/1813407328 shutdown_connections 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 --2- 172.21.15.38:0/1813407328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a8105d80 0x7fe8a8106180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- 172.21.15.38:0/1813407328 >> 172.21.15.38:0/1813407328 conn(0x7fe8a8101af0 msgr2=0x7fe8a8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:49.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- 172.21.15.38:0/1813407328 shutdown_connections 2024-09-19T05:31:49.609 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- 172.21.15.38:0/1813407328 wait complete. 2024-09-19T05:31:49.609 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 Processor -- start 2024-09-19T05:31:49.612 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.849+0000 7fe8af294640 1 -- start start 2024-09-19T05:31:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8af294640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8af294640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81077c0 con 0x7fe8a819d960 2024-09-19T05:31:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8ad030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8ad030640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53238/0 (socket says 172.21.15.38:53238) 2024-09-19T05:31:49.613 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8ad030640 1 -- 172.21.15.38:0/1644542534 learned_addr learned my addr 172.21.15.38:0/1644542534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8ad030640 1 -- 172.21.15.38:0/1644542534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a819e2c0 con 0x7fe8a819d960 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8ad030640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009940 tx=0x7fe8a4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8a4037050 con 0x7fe8a819d960 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8a4043070 con 0x7fe8a819d960 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8a403e6b0 con 0x7fe8a819d960 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.850+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a819e550 con 0x7fe8a819d960 2024-09-19T05:31:49.614 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.851+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a1030 con 0x7fe8a819d960 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fe8a403c610 con 0x7fe8a819d960 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe89a7fc640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 0x7fe89403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8106180 con 0x7fe8a819d960 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8a406d100 con 0x7fe8a819d960 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe8ac82f640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 0x7fe89403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.852+0000 7fe8ac82f640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 0x7fe89403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098c0 tx=0x7fe89c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:49.615 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:48.857+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8a404d200 con 0x7fe8a819d960 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.035+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe8a8062630 con 0x7fe894039200 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.035+0000 7fe89a7fc640 1 -- 172.21.15.38:0/1644542534 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe8a8062630 con 0x7fe894039200 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 msgr2=0x7fe89403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 0x7fe89403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098c0 tx=0x7fe89c006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 msgr2=0x7fe8a819dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009940 tx=0x7fe8a4032ce0 comp rx=0 tx=0).stop 2024-09-19T05:31:49.616 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 shutdown_connections 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fe894039200 0x7fe89403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 --2- 172.21.15.38:0/1644542534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a819d960 0x7fe8a819dd80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 >> 172.21.15.38:0/1644542534 conn(0x7fe8a8101af0 msgr2=0x7fe8a81944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 shutdown_connections 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:49.038+0000 7fe8af294640 1 -- 172.21.15.38:0/1644542534 wait complete. 2024-09-19T05:31:49.617 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: [19/Sep/2024:05:31:49] ENGINE Bus STARTING 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: [19/Sep/2024:05:31:49] ENGINE Serving on http://172.21.15.38:8765 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: [19/Sep/2024:05:31:49] ENGINE Serving on https://172.21.15.38:7150 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: [19/Sep/2024:05:31:49] ENGINE Bus STARTED 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:49 smithi038 ceph-mon[31227]: [19/Sep/2024:05:31:49] ENGINE Error in HTTPServer.serve 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:31:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:31:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-19T05:31:49.896 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: Generating public/private rsa key pair. 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: Your identification has been saved in /tmp/tmpr286d9cb/key 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: Your public key has been saved in /tmp/tmpr286d9cb/key.pub 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: The key fingerprint is: 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: SHA256:r63hABmnnzX4y9cd7VYdrExmziwF8bcHsbA8MB65azo ceph-44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:50.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: The key's randomart image is: 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: +---[RSA 3072]----+ 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | +oo . | 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | ..*.o o | 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | . . ..=.+. | 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | = . . *.+.| 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | + . S .X .o+| 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | o + oo. *. =| 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | + oo.... o.| 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | +E=. . . o| 2024-09-19T05:31:50.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: | *+. . | 2024-09-19T05:31:50.852 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:31:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: +----[SHA256]-----+ 2024-09-19T05:31:51.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.008+0000 7fa72345b640 1 Processor -- start 2024-09-19T05:31:51.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.008+0000 7fa72345b640 1 -- start start 2024-09-19T05:31:51.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.008+0000 7fa72345b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:51.133 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.008+0000 7fa72345b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c07c7f0 con 0x7fa71c07de00 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.009+0000 7fa7211f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.009+0000 7fa7211f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53246/0 (socket says 172.21.15.38:53246) 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.009+0000 7fa7211f7640 1 -- 172.21.15.38:0/4140661252 learned_addr learned my addr 172.21.15.38:0/4140661252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.009+0000 7fa7211f7640 1 -- 172.21.15.38:0/4140661252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c07c970 con 0x7fa71c07de00 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa7211f7640 1 --2- 172.21.15.38:0/4140661252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa718009820 tx=0x7fa71802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d893fc3254d82cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa713fff640 1 -- 172.21.15.38:0/4140661252 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718037070 con 0x7fa71c07de00 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa713fff640 1 -- 172.21.15.38:0/4140661252 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa718032900 con 0x7fa71c07de00 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa713fff640 1 -- 172.21.15.38:0/4140661252 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718030850 con 0x7fa71c07de00 2024-09-19T05:31:51.134 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa72345b640 1 -- 172.21.15.38:0/4140661252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 msgr2=0x7fa71c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.010+0000 7fa72345b640 1 --2- 172.21.15.38:0/4140661252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa718009820 tx=0x7fa71802a270 comp rx=0 tx=0).stop 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 -- 172.21.15.38:0/4140661252 shutdown_connections 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 --2- 172.21.15.38:0/4140661252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c07c2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 -- 172.21.15.38:0/4140661252 >> 172.21.15.38:0/4140661252 conn(0x7fa71c102ce0 msgr2=0x7fa71c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 -- 172.21.15.38:0/4140661252 shutdown_connections 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 -- 172.21.15.38:0/4140661252 wait complete. 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 Processor -- start 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 -- start start 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.011+0000 7fa72345b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa72345b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c10db40 con 0x7fa71c07de00 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7211f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:51.135 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7211f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53256/0 (socket says 172.21.15.38:53256) 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7211f7640 1 -- 172.21.15.38:0/4068293955 learned_addr learned my addr 172.21.15.38:0/4068293955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7211f7640 1 -- 172.21.15.38:0/4068293955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c19dbf0 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7211f7640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa718032ce0 tx=0x7fa718032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718042070 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7180308c0 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c19de80 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718037040 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.012+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c1a0b70 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.013+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa718045070 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.013+0000 7fa7127fc640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 0x7fa70803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.013+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c07c330 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.013+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa71806c320 con 0x7fa71c07de00 2024-09-19T05:31:51.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.014+0000 7fa7209f6640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 0x7fa70803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.014+0000 7fa7209f6640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 0x7fa70803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098c0 tx=0x7fa70c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.018+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa71804b170 con 0x7fa71c07de00 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.194+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa71c103f80 con 0x7fa7080391b0 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.542+0000 7fa7127fc640 1 -- 172.21.15.38:0/4068293955 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa71c103f80 con 0x7fa7080391b0 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 msgr2=0x7fa70803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 0x7fa70803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098c0 tx=0x7fa70c006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 msgr2=0x7fa71c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa718032ce0 tx=0x7fa718032d10 comp rx=0 tx=0).stop 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 shutdown_connections 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa7080391b0 0x7fa70803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 --2- 172.21.15.38:0/4068293955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa71c07de00 0x7fa71c19d6b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 >> 172.21.15.38:0/4068293955 conn(0x7fa71c102ce0 msgr2=0x7fa71c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:51.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 shutdown_connections 2024-09-19T05:31:51.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:50.544+0000 7fa72345b640 1 -- 172.21.15.38:0/4068293955 wait complete. 2024-09-19T05:31:51.704 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:51 smithi038 ceph-mon[31227]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:51.704 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:51 smithi038 ceph-mon[31227]: Generating ssh key... 2024-09-19T05:31:51.705 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:51 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:51.705 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:51 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.512+0000 7f931113d640 1 Processor -- start 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.512+0000 7f931113d640 1 -- start start 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f931113d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f931113d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f930c107ee0 con 0x7f930c107510 2024-09-19T05:31:52.291 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53272/0 (socket says 172.21.15.38:53272) 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f930ad9d640 1 -- 172.21.15.38:0/653748584 learned_addr learned my addr 172.21.15.38:0/653748584 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.513+0000 7f930ad9d640 1 -- 172.21.15.38:0/653748584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1086c0 con 0x7f930c107510 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.514+0000 7f930ad9d640 1 --2- 172.21.15.38:0/653748584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009820 tx=0x7f92fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59aa0b69ef29fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.514+0000 7f9309d9b640 1 -- 172.21.15.38:0/653748584 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92fc037070 con 0x7f930c107510 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.514+0000 7f9309d9b640 1 -- 172.21.15.38:0/653748584 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92fc032900 con 0x7f930c107510 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.514+0000 7f9309d9b640 1 -- 172.21.15.38:0/653748584 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92fc030850 con 0x7f930c107510 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 -- 172.21.15.38:0/653748584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 msgr2=0x7f930c107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 --2- 172.21.15.38:0/653748584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009820 tx=0x7f92fc02a270 comp rx=0 tx=0).stop 2024-09-19T05:31:52.292 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 -- 172.21.15.38:0/653748584 shutdown_connections 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 --2- 172.21.15.38:0/653748584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c107910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 -- 172.21.15.38:0/653748584 >> 172.21.15.38:0/653748584 conn(0x7f930c102ce0 msgr2=0x7f930c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 -- 172.21.15.38:0/653748584 shutdown_connections 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.515+0000 7f931113d640 1 -- 172.21.15.38:0/653748584 wait complete. 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f931113d640 1 Processor -- start 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f931113d640 1 -- start start 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f931113d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f931113d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f930c10ba70 con 0x7f930c107510 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53288/0 (socket says 172.21.15.38:53288) 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.516+0000 7f930ad9d640 1 -- 172.21.15.38:0/709586143 learned_addr learned my addr 172.21.15.38:0/709586143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f930ad9d640 1 -- 172.21.15.38:0/709586143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c19ba20 con 0x7f930c107510 2024-09-19T05:31:52.293 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f930ad9d640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009940 tx=0x7f92fc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92fc042070 con 0x7f930c107510 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c19bcb0 con 0x7f930c107510 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92fc0308c0 con 0x7f930c107510 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92fc037040 con 0x7f930c107510 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.517+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c19e9a0 con 0x7f930c107510 2024-09-19T05:31:52.294 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.518+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f92fc045070 con 0x7f930c107510 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.518+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c10bbf0 con 0x7f930c107510 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.518+0000 7f92f3fff640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 0x7f92f403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.518+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92fc06c510 con 0x7f930c107510 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.519+0000 7f930a59c640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 0x7f92f403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.519+0000 7f930a59c640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 0x7f92f403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93000098c0 tx=0x7f9300006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.523+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92fc02eca0 con 0x7f930c107510 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.702+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f930c103f30 con 0x7f92f40391b0 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.703+0000 7f92f3fff640 1 -- 172.21.15.38:0/709586143 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f930c103f30 con 0x7f92f40391b0 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 msgr2=0x7f92f403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:52.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 0x7f92f403b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f93000098c0 tx=0x7f9300006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 msgr2=0x7f930c19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009940 tx=0x7f92fc032ce0 comp rx=0 tx=0).stop 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 shutdown_connections 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f92f40391b0 0x7f92f403b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 --2- 172.21.15.38:0/709586143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f930c107510 0x7f930c19b4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.705+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 >> 172.21.15.38:0/709586143 conn(0x7f930c102ce0 msgr2=0x7f930c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.706+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 shutdown_connections 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:51.706+0000 7f931113d640 1 -- 172.21.15.38:0/709586143 wait complete. 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:31:52.296 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T05:31:52.299 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-09-19T05:31:52.870 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:52 smithi038 ceph-mon[31227]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:53 smithi038 ceph-mon[31227]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:55.092 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:54 smithi038 ceph-mon[31227]: Deploying cephadm binary to smithi038 2024-09-19T05:31:57.303 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.677+0000 7f414cc65640 1 Processor -- start 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.677+0000 7f414cc65640 1 -- start start 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414cc65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414cc65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4148075900 con 0x7f4148105d80 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53300/0 (socket says 172.21.15.38:53300) 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414659c640 1 -- 172.21.15.38:0/3937008988 learned_addr learned my addr 172.21.15.38:0/3937008988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:57.304 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.678+0000 7f414659c640 1 -- 172.21.15.38:0/3937008988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41481066c0 con 0x7f4148105d80 2024-09-19T05:31:57.305 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.679+0000 7f414659c640 1 --2- 172.21.15.38:0/3937008988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4138009820 tx=0x7f413802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa51fe5f30f6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.679+0000 7f414559a640 1 -- 172.21.15.38:0/3937008988 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138037070 con 0x7f4148105d80 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.679+0000 7f414559a640 1 -- 172.21.15.38:0/3937008988 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4138032900 con 0x7f4148105d80 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.679+0000 7f414559a640 1 -- 172.21.15.38:0/3937008988 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138030850 con 0x7f4148105d80 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 -- 172.21.15.38:0/3937008988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 msgr2=0x7f4148106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 --2- 172.21.15.38:0/3937008988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4138009820 tx=0x7f413802a270 comp rx=0 tx=0).stop 2024-09-19T05:31:57.310 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 -- 172.21.15.38:0/3937008988 shutdown_connections 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 --2- 172.21.15.38:0/3937008988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f4148106180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 -- 172.21.15.38:0/3937008988 >> 172.21.15.38:0/3937008988 conn(0x7f4148101af0 msgr2=0x7f4148103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 -- 172.21.15.38:0/3937008988 shutdown_connections 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.680+0000 7f414cc65640 1 -- 172.21.15.38:0/3937008988 wait complete. 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414cc65640 1 Processor -- start 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414cc65640 1 -- start start 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414cc65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414cc65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f41481077c0 con 0x7f4148105d80 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53308/0 (socket says 172.21.15.38:53308) 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414659c640 1 -- 172.21.15.38:0/3426666089 learned_addr learned my addr 172.21.15.38:0/3426666089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.681+0000 7f414659c640 1 -- 172.21.15.38:0/3426666089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414819dce0 con 0x7f4148105d80 2024-09-19T05:31:57.311 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f414659c640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41380097f0 tx=0x7f4138030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138037040 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4138040070 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414819df70 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f413803a390 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.682+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41481a0c60 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.683+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4138032900 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.683+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4148106180 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.683+0000 7f413f7fe640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 0x7f413403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.683+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f413806c7a0 con 0x7f4148105d80 2024-09-19T05:31:57.312 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.683+0000 7f4145d9b640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 0x7f413403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.684+0000 7f4145d9b640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 0x7f413403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f41300098c0 tx=0x7f4130006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.688+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4138040ce0 con 0x7f4148105d80 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:52.868+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) -- 0x7f4148062630 con 0x7f41340391b0 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.718+0000 7f413f7fe640 1 -- 172.21.15.38:0/3426666089 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4148062630 con 0x7f41340391b0 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 msgr2=0x7f413403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 0x7f413403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f41300098c0 tx=0x7f4130006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 msgr2=0x7f414819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41380097f0 tx=0x7f4138030d80 comp rx=0 tx=0).stop 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 shutdown_connections 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f41340391b0 0x7f413403b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 --2- 172.21.15.38:0/3426666089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4148105d80 0x7f414819d7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:57.313 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 >> 172.21.15.38:0/3426666089 conn(0x7f4148101af0 msgr2=0x7f4148191aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:57.314 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 shutdown_connections 2024-09-19T05:31:57.314 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:56.721+0000 7f414cc65640 1 -- 172.21.15.38:0/3426666089 wait complete. 2024-09-19T05:31:57.314 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mon service... 2024-09-19T05:31:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:57 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:31:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:57 smithi038 ceph-mon[31227]: Added host smithi038 2024-09-19T05:31:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:57 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.702+0000 7f4cb4ad9640 1 Processor -- start 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.702+0000 7f4cb4ad9640 1 -- start start 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.702+0000 7f4cb4ad9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.702+0000 7f4cb4ad9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb0107ee0 con 0x7f4cb0107510 2024-09-19T05:31:58.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.703+0000 7f4cae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.703+0000 7f4cae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47426/0 (socket says 172.21.15.38:47426) 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.703+0000 7f4cae59c640 1 -- 172.21.15.38:0/1056693298 learned_addr learned my addr 172.21.15.38:0/1056693298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.703+0000 7f4cae59c640 1 -- 172.21.15.38:0/1056693298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb01086c0 con 0x7f4cb0107510 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cae59c640 1 --2- 172.21.15.38:0/1056693298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009820 tx=0x7f4ca002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794261ef854fa25b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cadd9b640 1 -- 172.21.15.38:0/1056693298 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ca0037070 con 0x7f4cb0107510 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cadd9b640 1 -- 172.21.15.38:0/1056693298 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ca0032900 con 0x7f4cb0107510 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cadd9b640 1 -- 172.21.15.38:0/1056693298 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ca0030850 con 0x7f4cb0107510 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/1056693298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 msgr2=0x7f4cb0107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/1056693298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009820 tx=0x7f4ca002a270 comp rx=0 tx=0).stop 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/1056693298 shutdown_connections 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/1056693298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb0107910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/1056693298 >> 172.21.15.38:0/1056693298 conn(0x7f4cb0102ce0 msgr2=0x7f4cb0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:58.718 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/1056693298 shutdown_connections 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.704+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/1056693298 wait complete. 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.705+0000 7f4cb4ad9640 1 Processor -- start 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.705+0000 7f4cb4ad9640 1 -- start start 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.705+0000 7f4cb4ad9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.705+0000 7f4cb4ad9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb010ba70 con 0x7f4cb0107510 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47432/0 (socket says 172.21.15.38:47432) 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cae59c640 1 -- 172.21.15.38:0/779515800 learned_addr learned my addr 172.21.15.38:0/779515800 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cae59c640 1 -- 172.21.15.38:0/779515800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb007e080 con 0x7f4cb0107510 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cae59c640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009940 tx=0x7f4ca0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ca0042070 con 0x7f4cb0107510 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4ca00308c0 con 0x7f4cb0107510 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb007c270 con 0x7f4cb0107510 2024-09-19T05:31:58.719 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ca0037040 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.706+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb007c7b0 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.707+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4ca0045070 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.707+0000 7f4ca77fe640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 0x7f4c9c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.707+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb0107990 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.707+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ca006c4a0 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.707+0000 7f4ca7fff640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 0x7f4c9c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.708+0000 7f4ca7fff640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 0x7f4c9c03b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098c0 tx=0x7f4c98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.712+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ca0045360 con 0x7f4cb0107510 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.906+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4cb0103f30 con 0x7f4c9c039200 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.912+0000 7f4ca77fe640 1 -- 172.21.15.38:0/779515800 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4cb0103f30 con 0x7f4c9c039200 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.914+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 msgr2=0x7f4c9c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:58.720 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.914+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 0x7f4c9c03b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098c0 tx=0x7f4c98006db0 comp rx=0 tx=0).stop 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.914+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 msgr2=0x7f4cb007db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.914+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009940 tx=0x7f4ca0032ce0 comp rx=0 tx=0).stop 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 shutdown_connections 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4c9c039200 0x7f4c9c03b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 --2- 172.21.15.38:0/779515800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cb0107510 0x7f4cb007db40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 >> 172.21.15.38:0/779515800 conn(0x7f4cb0102ce0 msgr2=0x7f4cb0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 shutdown_connections 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:57.915+0000 7f4cb4ad9640 1 -- 172.21.15.38:0/779515800 wait complete. 2024-09-19T05:31:58.721 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mgr service... 2024-09-19T05:31:59.034 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:58 smithi038 ceph-mon[31227]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:31:59.035 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:58 smithi038 ceph-mon[31227]: Saving service mon spec with placement count:5 2024-09-19T05:31:59.035 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:31:58 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:32:00.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T05:32:00.295 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.372+0000 7f4d33f43640 1 Processor -- start 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.372+0000 7f4d33f43640 1 -- start start 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d33f43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d33f43640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c105fc0 con 0x7f4d2c107a80 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d31cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d31cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47446/0 (socket says 172.21.15.38:47446) 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d31cdf640 1 -- 172.21.15.38:0/1903370052 learned_addr learned my addr 172.21.15.38:0/1903370052 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.373+0000 7f4d31cdf640 1 -- 172.21.15.38:0/1903370052 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c106140 con 0x7f4d2c107a80 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.374+0000 7f4d31cdf640 1 --2- 172.21.15.38:0/1903370052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009820 tx=0x7f4d2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f7d8ee7ba87c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.374+0000 7f4d30cdd640 1 -- 172.21.15.38:0/1903370052 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28037070 con 0x7f4d2c107a80 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.374+0000 7f4d30cdd640 1 -- 172.21.15.38:0/1903370052 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d28032900 con 0x7f4d2c107a80 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.374+0000 7f4d30cdd640 1 -- 172.21.15.38:0/1903370052 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28030850 con 0x7f4d2c107a80 2024-09-19T05:32:00.296 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 -- 172.21.15.38:0/1903370052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 msgr2=0x7f4d2c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1903370052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009820 tx=0x7f4d2802a270 comp rx=0 tx=0).stop 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 -- 172.21.15.38:0/1903370052 shutdown_connections 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1903370052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c107e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 -- 172.21.15.38:0/1903370052 >> 172.21.15.38:0/1903370052 conn(0x7f4d2c101af0 msgr2=0x7f4d2c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 -- 172.21.15.38:0/1903370052 shutdown_connections 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 -- 172.21.15.38:0/1903370052 wait complete. 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.375+0000 7f4d33f43640 1 Processor -- start 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d33f43640 1 -- start start 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d33f43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d33f43640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10a950 con 0x7f4d2c107a80 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d31cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d31cdf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47450/0 (socket says 172.21.15.38:47450) 2024-09-19T05:32:00.297 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d31cdf640 1 -- 172.21.15.38:0/1379182349 learned_addr learned my addr 172.21.15.38:0/1379182349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d31cdf640 1 -- 172.21.15.38:0/1379182349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c19e300 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d31cdf640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009940 tx=0x7f4d28032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.376+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28042070 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.377+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d280308c0 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.377+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c19e4d0 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.377+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28037040 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.377+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c19e8f0 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.377+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4d28045070 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.378+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d100052c0 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.378+0000 7f4d22ffd640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 0x7f4d1803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.378+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d2806c2e0 con 0x7f4d2c107a80 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.378+0000 7f4d314de640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 0x7f4d1803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:00.298 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.378+0000 7f4d314de640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 0x7f4d1803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0098c0 tx=0x7f4d1c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.383+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d28045520 con 0x7f4d2c107a80 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.573+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4d10000bf0 con 0x7f4d18039200 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.578+0000 7f4d22ffd640 1 -- 172.21.15.38:0/1379182349 <== mgr.14118 v2:172.21.15.38:6800/3555821050 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4d10000bf0 con 0x7f4d18039200 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 msgr2=0x7f4d1803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 0x7f4d1803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0098c0 tx=0x7f4d1c006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 msgr2=0x7f4d2c075460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009940 tx=0x7f4d28032ce0 comp rx=0 tx=0).stop 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 shutdown_connections 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4d18039200 0x7f4d1803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 --2- 172.21.15.38:0/1379182349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d2c107a80 0x7f4d2c075460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 >> 172.21.15.38:0/1379182349 conn(0x7f4d2c101af0 msgr2=0x7f4d2c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 shutdown_connections 2024-09-19T05:32:00.299 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:31:59.580+0000 7f4d33f43640 1 -- 172.21.15.38:0/1379182349 wait complete. 2024-09-19T05:32:00.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:00 smithi038 ceph-mon[31227]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:00.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:00 smithi038 ceph-mon[31227]: Saving service mgr spec with placement count:2 2024-09-19T05:32:00.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:00 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:32:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa49a255640 1 Processor -- start 2024-09-19T05:32:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa49a255640 1 -- start start 2024-09-19T05:32:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa49a255640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa49a255640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa494106180 con 0x7fa4941057b0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47456/0 (socket says 172.21.15.38:47456) 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.787+0000 7fa4937fe640 1 -- 172.21.15.38:0/2902472314 learned_addr learned my addr 172.21.15.38:0/2902472314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.788+0000 7fa4937fe640 1 -- 172.21.15.38:0/2902472314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494108790 con 0x7fa4941057b0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa4937fe640 1 --2- 172.21.15.38:0/2902472314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa484009820 tx=0x7fa48402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b946879ea5e05cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa492ffd640 1 -- 172.21.15.38:0/2902472314 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484037070 con 0x7fa4941057b0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa492ffd640 1 -- 172.21.15.38:0/2902472314 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa484032900 con 0x7fa4941057b0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa492ffd640 1 -- 172.21.15.38:0/2902472314 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484030850 con 0x7fa4941057b0 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa49a255640 1 -- 172.21.15.38:0/2902472314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 msgr2=0x7fa494105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa49a255640 1 --2- 172.21.15.38:0/2902472314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa484009820 tx=0x7fa48402a270 comp rx=0 tx=0).stop 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa49a255640 1 -- 172.21.15.38:0/2902472314 shutdown_connections 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa49a255640 1 --2- 172.21.15.38:0/2902472314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494105bb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.789+0000 7fa49a255640 1 -- 172.21.15.38:0/2902472314 >> 172.21.15.38:0/2902472314 conn(0x7fa494101360 msgr2=0x7fa494103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.790+0000 7fa49a255640 1 -- 172.21.15.38:0/2902472314 shutdown_connections 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.790+0000 7fa49a255640 1 -- 172.21.15.38:0/2902472314 wait complete. 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.790+0000 7fa49a255640 1 Processor -- start 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.790+0000 7fa49a255640 1 -- start start 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa49a255640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa49a255640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa49410b9c0 con 0x7fa4941057b0 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47464/0 (socket says 172.21.15.38:47464) 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4937fe640 1 -- 172.21.15.38:0/1439834339 learned_addr learned my addr 172.21.15.38:0/1439834339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4937fe640 1 -- 172.21.15.38:0/1439834339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494108550 con 0x7fa4941057b0 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4937fe640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097f0 tx=0x7fa484030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa484037040 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa494106740 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa484040070 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.791+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48403a390 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.792+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa494106c20 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa484032900 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa494105c30 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa4917fa640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 0x7fa48003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa48406c740 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa48affd640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 0x7fa48003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.793+0000 7fa48affd640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 0x7fa48003b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0098c0 tx=0x7fa47c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.798+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa48406ecc0 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.975+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa494107460 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.982+0000 7fa4917fa640 1 -- 172.21.15.38:0/1439834339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fa48403a630 con 0x7fa4941057b0 2024-09-19T05:32:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.984+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 msgr2=0x7fa48003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.984+0000 7fa49a255640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 0x7fa48003b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0098c0 tx=0x7fa47c006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.984+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 msgr2=0x7fa494108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.984+0000 7fa49a255640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa4840097f0 tx=0x7fa484030d80 comp rx=0 tx=0).stop 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 shutdown_connections 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7fa4800391b0 0x7fa48003b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 --2- 172.21.15.38:0/1439834339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4941057b0 0x7fa494108010 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 >> 172.21.15.38:0/1439834339 conn(0x7fa494101360 msgr2=0x7fa494101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 shutdown_connections 2024-09-19T05:32:01.850 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:00.985+0000 7fa49a255640 1 -- 172.21.15.38:0/1439834339 wait complete. 2024-09-19T05:32:02.134 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:01 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1439834339' entity='client.admin' 2024-09-19T05:32:02.134 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:01 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:32:02.134 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:01 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de46bf640 1 Processor -- start 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de46bf640 1 -- start start 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de46bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de46bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc107ee0 con 0x7f4ddc107510 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de245b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de245b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47474/0 (socket says 172.21.15.38:47474) 2024-09-19T05:32:03.114 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.213+0000 7f4de245b640 1 -- 172.21.15.38:0/4209563941 learned_addr learned my addr 172.21.15.38:0/4209563941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.214+0000 7f4de245b640 1 -- 172.21.15.38:0/4209563941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc1086c0 con 0x7f4ddc107510 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.214+0000 7f4de245b640 1 --2- 172.21.15.38:0/4209563941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009820 tx=0x7f4dd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225ec96570758cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de1459640 1 -- 172.21.15.38:0/4209563941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8037070 con 0x7f4ddc107510 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de1459640 1 -- 172.21.15.38:0/4209563941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd8032920 con 0x7f4ddc107510 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de1459640 1 -- 172.21.15.38:0/4209563941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8030810 con 0x7f4ddc107510 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 -- 172.21.15.38:0/4209563941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 msgr2=0x7f4ddc107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 --2- 172.21.15.38:0/4209563941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009820 tx=0x7f4dd802a270 comp rx=0 tx=0).stop 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 -- 172.21.15.38:0/4209563941 shutdown_connections 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 --2- 172.21.15.38:0/4209563941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc107910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 -- 172.21.15.38:0/4209563941 >> 172.21.15.38:0/4209563941 conn(0x7f4ddc102ce0 msgr2=0x7f4ddc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 -- 172.21.15.38:0/4209563941 shutdown_connections 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.215+0000 7f4de46bf640 1 -- 172.21.15.38:0/4209563941 wait complete. 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de46bf640 1 Processor -- start 2024-09-19T05:32:03.115 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de46bf640 1 -- start start 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de46bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de46bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc10ba70 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de245b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de245b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47488/0 (socket says 172.21.15.38:47488) 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.216+0000 7f4de245b640 1 -- 172.21.15.38:0/1300986422 learned_addr learned my addr 172.21.15.38:0/1300986422 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4de245b640 1 -- 172.21.15.38:0/1300986422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc07e080 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4de245b640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8030560 tx=0x7f4dd8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd8037040 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd8040070 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc07c270 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dd803a740 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.217+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc07c7b0 con 0x7f4ddc107510 2024-09-19T05:32:03.116 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.218+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4dd803a9b0 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.218+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc107990 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.218+0000 7f4dd37fe640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 0x7f4dc803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.218+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4dd806c940 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.218+0000 7f4de1c5a640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 0x7f4dc803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.219+0000 7f4de1c5a640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 0x7f4dc803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098c0 tx=0x7f4dcc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.223+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4dd804a200 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.402+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4ddc07ca30 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.405+0000 7f4dd37fe640 1 -- 172.21.15.38:0/1300986422 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4dd8031e80 con 0x7f4ddc107510 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 msgr2=0x7f4dc803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 0x7f4dc803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0098c0 tx=0x7f4dcc006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 msgr2=0x7f4ddc07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8030560 tx=0x7f4dd8032670 comp rx=0 tx=0).stop 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 shutdown_connections 2024-09-19T05:32:03.117 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f4dc80391b0 0x7f4dc803b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:03.118 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 --2- 172.21.15.38:0/1300986422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ddc107510 0x7f4ddc07db40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:03.118 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 >> 172.21.15.38:0/1300986422 conn(0x7f4ddc102ce0 msgr2=0x7f4ddc103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:03.118 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 shutdown_connections 2024-09-19T05:32:03.118 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:02.407+0000 7f4de46bf640 1 -- 172.21.15.38:0/1300986422 wait complete. 2024-09-19T05:32:03.118 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-09-19T05:32:03.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:03 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1300986422' entity='client.admin' 2024-09-19T05:32:04.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3193649816' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T05:32:04.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-mon[31227]: from='mgr.14118 172.21.15.38:0/482906797' entity='mgr.x' 2024-09-19T05:32:04.565 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ignoring --setuser ceph since I am not root 2024-09-19T05:32:04.565 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ignoring --setgroup ceph since I am not root 2024-09-19T05:32:04.565 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:04.562+0000 7f3967534040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:32:04.850 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:04.622+0000 7f3967534040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:32:04.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:04.779+0000 7f3967534040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:32:04.851 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:04.847+0000 7f3967534040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:32:04.981 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.511+0000 7f09e54e6640 1 Processor -- start 2024-09-19T05:32:04.981 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.511+0000 7f09e54e6640 1 -- start start 2024-09-19T05:32:04.981 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09e54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09e54e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010a840 con 0x7f09e0107f10 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47492/0 (socket says 172.21.15.38:47492) 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09deffd640 1 -- 172.21.15.38:0/1504190118 learned_addr learned my addr 172.21.15.38:0/1504190118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.512+0000 7f09deffd640 1 -- 172.21.15.38:0/1504190118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e010a9c0 con 0x7f09e0107f10 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.513+0000 7f09deffd640 1 --2- 172.21.15.38:0/1504190118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009820 tx=0x7f09d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15348d07855bf212 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.513+0000 7f09ddffb640 1 -- 172.21.15.38:0/1504190118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d0037070 con 0x7f09e0107f10 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.513+0000 7f09ddffb640 1 -- 172.21.15.38:0/1504190118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d0032920 con 0x7f09e0107f10 2024-09-19T05:32:04.982 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.513+0000 7f09ddffb640 1 -- 172.21.15.38:0/1504190118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d0030810 con 0x7f09e0107f10 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 -- 172.21.15.38:0/1504190118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 msgr2=0x7f09e010a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 --2- 172.21.15.38:0/1504190118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009820 tx=0x7f09d002a270 comp rx=0 tx=0).stop 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 -- 172.21.15.38:0/1504190118 shutdown_connections 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 --2- 172.21.15.38:0/1504190118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e010a300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 -- 172.21.15.38:0/1504190118 >> 172.21.15.38:0/1504190118 conn(0x7f09e0101af0 msgr2=0x7f09e0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 -- 172.21.15.38:0/1504190118 shutdown_connections 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 -- 172.21.15.38:0/1504190118 wait complete. 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.514+0000 7f09e54e6640 1 Processor -- start 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09e54e6640 1 -- start start 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09e54e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09e54e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010b9e0 con 0x7f09e0107f10 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:04.983 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09deffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47502/0 (socket says 172.21.15.38:47502) 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09deffd640 1 -- 172.21.15.38:0/3193649816 learned_addr learned my addr 172.21.15.38:0/3193649816 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09deffd640 1 -- 172.21.15.38:0/3193649816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e019bb10 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.515+0000 7f09deffd640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097f0 tx=0x7f09d00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.516+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d0037040 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.516+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d0030cc0 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.516+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e019bda0 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.516+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d003a6b0 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.516+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e019ea90 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f09d003a850 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09c7fff640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 0x7f09c803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09d006c790 con 0x7f09e0107f10 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09de7fc640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 0x7f09c803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:04.984 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0108370 con 0x7f09e0107f10 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.517+0000 7f09de7fc640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 0x7f09c803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098c0 tx=0x7f09d4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.522+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09d002fd60 con 0x7f09e0107f10 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:03.747+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f09e019eeb0 con 0x7f09e0107f10 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.411+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f09d0044920 con 0x7f09e0107f10 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.412+0000 7f09c7fff640 1 -- 172.21.15.38:0/3193649816 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f09d0044400 con 0x7f09e0107f10 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.413+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 msgr2=0x7f09c803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.413+0000 7f09e54e6640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 0x7f09c803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098c0 tx=0x7f09d4006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.413+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 msgr2=0x7f09e019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.413+0000 7f09e54e6640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097f0 tx=0x7f09d00327b0 comp rx=0 tx=0).stop 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 shutdown_connections 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f09c8039200 0x7f09c803b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 --2- 172.21.15.38:0/3193649816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09e0107f10 0x7f09e019b5d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 >> 172.21.15.38:0/3193649816 conn(0x7f09e0101af0 msgr2=0x7f09e01026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:04.986 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 shutdown_connections 2024-09-19T05:32:04.986 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:04.414+0000 7f09e54e6640 1 -- 172.21.15.38:0/3193649816 wait complete. 2024-09-19T05:32:05.110 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:05.014+0000 7f3967534040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:32:05.361 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:05.163+0000 7f3967534040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:32:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3193649816' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T05:32:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-mon[31227]: mgrmap e9: x(active, since 18s) 2024-09-19T05:32:05.645 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:05.497+0000 7f3967534040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:32:05.645 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:05.561+0000 7f3967534040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:32:06.048 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:05.759+0000 7f3967534040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:32:06.325 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:32:06.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T05:32:06.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T05:32:06.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T05:32:06.332 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.439+0000 7f61775af640 1 Processor -- start 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.439+0000 7f61775af640 1 -- start start 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.439+0000 7f61775af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.439+0000 7f61775af640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6170075900 con 0x7f6170105d80 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.440+0000 7f617534b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.440+0000 7f617534b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47528/0 (socket says 172.21.15.38:47528) 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.440+0000 7f617534b640 1 -- 172.21.15.38:0/2143593116 learned_addr learned my addr 172.21.15.38:0/2143593116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.440+0000 7f617534b640 1 -- 172.21.15.38:0/2143593116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61701066c0 con 0x7f6170105d80 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.441+0000 7f617534b640 1 --2- 172.21.15.38:0/2143593116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6164009820 tx=0x7f616402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad832ac2470bece server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.441+0000 7f6163fff640 1 -- 172.21.15.38:0/2143593116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164037070 con 0x7f6170105d80 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.441+0000 7f6163fff640 1 -- 172.21.15.38:0/2143593116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6164032920 con 0x7f6170105d80 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.441+0000 7f6163fff640 1 -- 172.21.15.38:0/2143593116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164030810 con 0x7f6170105d80 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 -- 172.21.15.38:0/2143593116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 msgr2=0x7f6170106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 --2- 172.21.15.38:0/2143593116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6164009820 tx=0x7f616402a270 comp rx=0 tx=0).stop 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 -- 172.21.15.38:0/2143593116 shutdown_connections 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 --2- 172.21.15.38:0/2143593116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f6170106180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 -- 172.21.15.38:0/2143593116 >> 172.21.15.38:0/2143593116 conn(0x7f6170101af0 msgr2=0x7f6170103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 -- 172.21.15.38:0/2143593116 shutdown_connections 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 -- 172.21.15.38:0/2143593116 wait complete. 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.442+0000 7f61775af640 1 Processor -- start 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f61775af640 1 -- start start 2024-09-19T05:32:06.334 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f61775af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f61775af640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6170107800 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f617534b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f617534b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47540/0 (socket says 172.21.15.38:47540) 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f617534b640 1 -- 172.21.15.38:0/3728365346 learned_addr learned my addr 172.21.15.38:0/3728365346 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f617534b640 1 -- 172.21.15.38:0/3728365346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617019dcc0 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f617534b640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61640097f0 tx=0x7f61640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164037040 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.443+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6164030cc0 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.444+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617019df50 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.444+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616403a6b0 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.444+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a0c40 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f616403a8a0 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6170106180 con 0x7f6170105d80 2024-09-19T05:32:06.335 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f61627fc640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 0x7f615c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f616406c810 con 0x7f6170105d80 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f6174b4a640 1 -- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 msgr2=0x7f615c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.445+0000 7f6174b4a640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 0x7f615c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.450+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61640371e0 con 0x7f6170105d80 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.646+0000 7f6174b4a640 1 -- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 msgr2=0x7f615c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.646+0000 7f6174b4a640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 0x7f615c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.674+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f61701a0e10 con 0x7f6170105d80 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.675+0000 7f61627fc640 1 -- 172.21.15.38:0/3728365346 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f61640319f0 con 0x7f6170105d80 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 msgr2=0x7f615c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 0x7f615c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 msgr2=0x7f617019d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61640097f0 tx=0x7f61640327b0 comp rx=0 tx=0).stop 2024-09-19T05:32:06.336 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 shutdown_connections 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f615c039200 0x7f615c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 --2- 172.21.15.38:0/3728365346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6170105d80 0x7f617019d780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 >> 172.21.15.38:0/3728365346 conn(0x7f6170101af0 msgr2=0x7f6170191a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 shutdown_connections 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:05.677+0000 7f61775af640 1 -- 172.21.15.38:0/3728365346 wait complete. 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-19T05:32:06.337 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 9... 2024-09-19T05:32:06.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:06 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3728365346' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T05:32:06.972 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:06.691+0000 7f3967534040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:32:06.972 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:06.764+0000 7f3967534040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:32:07.633 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:07.509+0000 7f3967534040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:32:07.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:07.630+0000 7f3967534040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:32:07.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:07.779+0000 7f3967534040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:32:07.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:07.843+0000 7f3967534040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:32:08.301 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:07.998+0000 7f3967534040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:32:08.301 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.069+0000 7f3967534040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:32:08.301 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.133+0000 7f3967534040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:32:08.301 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.298+0000 7f3967534040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:32:08.573 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.367+0000 7f3967534040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:32:08.573 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.438+0000 7f3967534040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:32:08.573 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.526+0000 7f3967534040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:32:08.893 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.590+0000 7f3967534040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:32:08.894 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:08.653+0000 7f3967534040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:32:09.746 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:09.493+0000 7f3967534040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: Active manager daemon x restarted 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: Activating manager daemon x 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: mgrmap e10: x(active, starting, since 0.00576407s) 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: Manager daemon x is now available 2024-09-19T05:32:09.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:09.748 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:32:09.748 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:32:11.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-19T05:32:11.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T05:32:11.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T05:32:11.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-19T05:32:11.403 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a85065640 1 Processor -- start 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a85065640 1 -- start start 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a85065640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a85065640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80075900 con 0x7f7a80107f90 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47554/0 (socket says 172.21.15.38:47554) 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.763+0000 7f7a7ed9d640 1 -- 172.21.15.38:0/405504190 learned_addr learned my addr 172.21.15.38:0/405504190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.764+0000 7f7a7ed9d640 1 -- 172.21.15.38:0/405504190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8010a8c0 con 0x7f7a80107f90 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.764+0000 7f7a7ed9d640 1 --2- 172.21.15.38:0/405504190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009e80 tx=0x7f7a6802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a295ad1f978345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a7dd9b640 1 -- 172.21.15.38:0/405504190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68036070 con 0x7f7a80107f90 2024-09-19T05:32:11.404 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a7dd9b640 1 -- 172.21.15.38:0/405504190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a680318e0 con 0x7f7a80107f90 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a7dd9b640 1 -- 172.21.15.38:0/405504190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68007a00 con 0x7f7a80107f90 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 -- 172.21.15.38:0/405504190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 msgr2=0x7f7a8010a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 --2- 172.21.15.38:0/405504190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009e80 tx=0x7f7a6802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 -- 172.21.15.38:0/405504190 shutdown_connections 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 --2- 172.21.15.38:0/405504190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8010a380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 -- 172.21.15.38:0/405504190 >> 172.21.15.38:0/405504190 conn(0x7f7a80101af0 msgr2=0x7f7a80103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 -- 172.21.15.38:0/405504190 shutdown_connections 2024-09-19T05:32:11.405 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.765+0000 7f7a85065640 1 -- 172.21.15.38:0/405504190 wait complete. 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a85065640 1 Processor -- start 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a85065640 1 -- start start 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a85065640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a85065640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8010ba00 con 0x7f7a80107f90 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a7ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47558/0 (socket says 172.21.15.38:47558) 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.766+0000 7f7a7ed9d640 1 -- 172.21.15.38:0/2965203873 learned_addr learned my addr 172.21.15.38:0/2965203873 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a7ed9d640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8019bb20 con 0x7f7a80107f90 2024-09-19T05:32:11.406 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a7ed9d640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a6802acc0 tx=0x7f7a68031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a68036050 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a68037040 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8019bdb0 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a6803d8e0 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.767+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8019eaa0 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f7a68044020 con 0x7f7a80107f90 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a77fff640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:11.407 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7a6c03bdd0 con 0x7f7a6c039200 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a68042020 con 0x7f7a80107f90 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a7e59c640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 msgr2=0x7f7a6c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.768+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.969+0000 7f7a7e59c640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 msgr2=0x7f7a6c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:06.969+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:07.370+0000 7f7a7e59c640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 msgr2=0x7f7a6c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:11.408 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:07.370+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:08.171+0000 7f7a7e59c640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 msgr2=0x7f7a6c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3555821050 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:08.171+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:09.499+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f7a6806b910 con 0x7f7a80107f90 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:09.499+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 msgr2=0x7f7a6c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:09.499+0000 7f7a77fff640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.499+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f7a68044310 con 0x7f7a80107f90 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.499+0000 7f7a77fff640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 0x7f7a6c03f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.499+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7a6c03bdd0 con 0x7f7a6c03ccc0 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.500+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 0x7f7a6c03f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.500+0000 7f7a7e59c640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 0x7f7a6c03f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a70003980 tx=0x7f7a70006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:11.409 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.501+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7a6c03bdd0 con 0x7f7a6c03ccc0 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.508+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7a8019c290 con 0x7f7a6c03ccc0 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a77fff640 1 -- 172.21.15.38:0/2965203873 <== mgr.14150 v2:172.21.15.38:6800/4137587885 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7a8019c290 con 0x7f7a6c03ccc0 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 msgr2=0x7f7a6c03f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 0x7f7a6c03f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a70003980 tx=0x7f7a70006e10 comp rx=0 tx=0).stop 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 msgr2=0x7f7a8019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a6802acc0 tx=0x7f7a68031cc0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 shutdown_connections 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7a6c03ccc0 0x7f7a6c03f0b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:6800/3555821050,v1:172.21.15.38:6801/3555821050] conn(0x7f7a6c039200 0x7f7a6c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 --2- 172.21.15.38:0/2965203873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a80107f90 0x7f7a8019b5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 >> 172.21.15.38:0/2965203873 conn(0x7f7a80101af0 msgr2=0x7f7a801026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 shutdown_connections 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:10.509+0000 7f7a85065640 1 -- 172.21.15.38:0/2965203873 wait complete. 2024-09-19T05:32:11.410 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 9 is available 2024-09-19T05:32:11.411 INFO:teuthology.orchestra.run.smithi038.stdout:Generating a dashboard self-signed certificate... 2024-09-19T05:32:11.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: mgrmap e11: x(active, since 1.00676s) 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: [19/Sep/2024:05:32:11] ENGINE Bus STARTING 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: [19/Sep/2024:05:32:11] ENGINE Serving on http://172.21.15.38:8765 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: [19/Sep/2024:05:32:11] ENGINE Serving on https://172.21.15.38:7150 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: [19/Sep/2024:05:32:11] ENGINE Bus STARTED 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-mon[31227]: [19/Sep/2024:05:32:11] ENGINE Error in HTTPServer.serve 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-19T05:32:11.722 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-19T05:32:11.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:11.460+0000 7f38ccffd640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:32:11] ENGINE Error in HTTPServer.serve 2024-09-19T05:32:11.724 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: Traceback (most recent call last): 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._connections.run(self.expiration_interval) 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._run(expiration_interval) 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: s = self.context.wrap_socket( 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: return self.sslsocket_class._create( 2024-09-19T05:32:11.725 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:32:11.726 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self.do_handshake() 2024-09-19T05:32:11.726 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:32:11.726 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: self._sslobj.do_handshake() 2024-09-19T05:32:11.726 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:32:11.726 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:32:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:32:12.633 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T05:32:12.633 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.802+0000 7f9f8db17640 1 Processor -- start 2024-09-19T05:32:12.633 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.802+0000 7f9f8db17640 1 -- start start 2024-09-19T05:32:12.633 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.802+0000 7f9f8db17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:12.633 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.802+0000 7f9f8db17640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88105fc0 con 0x7f9f88107a80 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.803+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.803+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41250/0 (socket says 172.21.15.38:41250) 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.803+0000 7f9f877fe640 1 -- 172.21.15.38:0/4055062564 learned_addr learned my addr 172.21.15.38:0/4055062564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.803+0000 7f9f877fe640 1 -- 172.21.15.38:0/4055062564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f88106140 con 0x7f9f88107a80 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.804+0000 7f9f877fe640 1 --2- 172.21.15.38:0/4055062564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009820 tx=0x7f9f7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf99ead7c5e0965 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.804+0000 7f9f867fc640 1 -- 172.21.15.38:0/4055062564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f78037070 con 0x7f9f88107a80 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.804+0000 7f9f867fc640 1 -- 172.21.15.38:0/4055062564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78032920 con 0x7f9f88107a80 2024-09-19T05:32:12.634 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.804+0000 7f9f867fc640 1 -- 172.21.15.38:0/4055062564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f78030810 con 0x7f9f88107a80 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 -- 172.21.15.38:0/4055062564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 msgr2=0x7f9f88107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 --2- 172.21.15.38:0/4055062564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009820 tx=0x7f9f7802a270 comp rx=0 tx=0).stop 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 -- 172.21.15.38:0/4055062564 shutdown_connections 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 --2- 172.21.15.38:0/4055062564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f88107e80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 -- 172.21.15.38:0/4055062564 >> 172.21.15.38:0/4055062564 conn(0x7f9f88101af0 msgr2=0x7f9f88103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 -- 172.21.15.38:0/4055062564 shutdown_connections 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 -- 172.21.15.38:0/4055062564 wait complete. 2024-09-19T05:32:12.635 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.805+0000 7f9f8db17640 1 Processor -- start 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f8db17640 1 -- start start 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f8db17640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f8db17640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810a950 con 0x7f9f88107a80 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41264/0 (socket says 172.21.15.38:41264) 2024-09-19T05:32:12.636 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f877fe640 1 -- 172.21.15.38:0/316528295 learned_addr learned my addr 172.21.15.38:0/316528295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:12.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.806+0000 7f9f877fe640 1 -- 172.21.15.38:0/316528295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8819dd80 con 0x7f9f88107a80 2024-09-19T05:32:12.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f877fe640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f78030560 tx=0x7f9f78032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:12.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f78037040 con 0x7f9f88107a80 2024-09-19T05:32:12.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78040070 con 0x7f9f88107a80 2024-09-19T05:32:12.637 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8819e010 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f7803a740 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.807+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a0d00 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.808+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f9f7803a8e0 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.808+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f88107f00 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.808+0000 7f9f84ff9640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 0x7f9f7003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.808+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9f7806c7b0 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.808+0000 7f9f86ffd640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 0x7f9f7003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.809+0000 7f9f86ffd640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 0x7f9f7003b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098c0 tx=0x7f9f7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.813+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f78044c70 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:11.862+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f9f7806ecc0 con 0x7f9f88107a80 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.008+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9f8802d910 con 0x7f9f70039140 2024-09-19T05:32:12.638 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.094+0000 7f9f84ff9640 1 -- 172.21.15.38:0/316528295 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9f8802d910 con 0x7f9f70039140 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 msgr2=0x7f9f7003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 0x7f9f7003b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098c0 tx=0x7f9f7c006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 msgr2=0x7f9f8819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f78030560 tx=0x7f9f78032670 comp rx=0 tx=0).stop 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 shutdown_connections 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f70039140 0x7f9f7003b600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 --2- 172.21.15.38:0/316528295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f88107a80 0x7f9f8819d840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 >> 172.21.15.38:0/316528295 conn(0x7f9f88101af0 msgr2=0x7f9f88102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 shutdown_connections 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:12.096+0000 7f9f8db17640 1 -- 172.21.15.38:0/316528295 wait complete. 2024-09-19T05:32:12.639 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-09-19T05:32:13.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:13.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:12 smithi038 ceph-mon[31227]: mgrmap e12: x(active, since 2s) 2024-09-19T05:32:13.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:12 smithi038 ceph-mon[31227]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:13.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:13.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:14.035 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:13 smithi038 ceph-mon[31227]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:14.035 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:14.514 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xsS5k2GnH3XXmvomXVC3DeWEqLysac8cgTyJDjr7Cknr1jz4F2YGm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726723933, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f18008d1640 1 Processor -- start 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f18008d1640 1 -- start start 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f18008d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f18008d1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc107ee0 con 0x7f17fc107510 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f17fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f17fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41272/0 (socket says 172.21.15.38:41272) 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.062+0000 7f17fa59c640 1 -- 172.21.15.38:0/3058907034 learned_addr learned my addr 172.21.15.38:0/3058907034 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.063+0000 7f17fa59c640 1 -- 172.21.15.38:0/3058907034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc1086c0 con 0x7f17fc107510 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.063+0000 7f17fa59c640 1 --2- 172.21.15.38:0/3058907034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009820 tx=0x7f17ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2678dc7ff7c5e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.063+0000 7f17f9d9b640 1 -- 172.21.15.38:0/3058907034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec037070 con 0x7f17fc107510 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f17f9d9b640 1 -- 172.21.15.38:0/3058907034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec032920 con 0x7f17fc107510 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f17f9d9b640 1 -- 172.21.15.38:0/3058907034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec030810 con 0x7f17fc107510 2024-09-19T05:32:14.515 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 -- 172.21.15.38:0/3058907034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 msgr2=0x7f17fc107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 --2- 172.21.15.38:0/3058907034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009820 tx=0x7f17ec02a270 comp rx=0 tx=0).stop 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 -- 172.21.15.38:0/3058907034 shutdown_connections 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 --2- 172.21.15.38:0/3058907034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc107910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 -- 172.21.15.38:0/3058907034 >> 172.21.15.38:0/3058907034 conn(0x7f17fc102ce0 msgr2=0x7f17fc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 -- 172.21.15.38:0/3058907034 shutdown_connections 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.064+0000 7f18008d1640 1 -- 172.21.15.38:0/3058907034 wait complete. 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f18008d1640 1 Processor -- start 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f18008d1640 1 -- start start 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f18008d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f18008d1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc10ba70 con 0x7f17fc107510 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f17fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:14.516 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f17fa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41286/0 (socket says 172.21.15.38:41286) 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.065+0000 7f17fa59c640 1 -- 172.21.15.38:0/1933891287 learned_addr learned my addr 172.21.15.38:0/1933891287 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f17fa59c640 1 -- 172.21.15.38:0/1933891287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19ba20 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f17fa59c640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17ec030560 tx=0x7f17ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec037040 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ec040070 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19bcb0 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17ec03a740 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.066+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc19e9a0 con 0x7f17fc107510 2024-09-19T05:32:14.517 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.067+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f17ec03a9b0 con 0x7f17fc107510 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.067+0000 7f17f3fff640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 0x7f17e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.067+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc10bbf0 con 0x7f17fc107510 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.067+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17ec06cbf0 con 0x7f17fc107510 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.067+0000 7f17f1d9b640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 0x7f17e803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.068+0000 7f17f1d9b640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 0x7f17e803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17e40098c0 tx=0x7f17e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.072+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f17ec042e30 con 0x7f17fc107510 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.264+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f17fc107990 con 0x7f17e8039200 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.514+0000 7f17f3fff640 1 -- 172.21.15.38:0/1933891287 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f17fc107990 con 0x7f17e8039200 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 msgr2=0x7f17e803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 0x7f17e803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17e40098c0 tx=0x7f17e4006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 msgr2=0x7f17fc19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17ec030560 tx=0x7f17ec032670 comp rx=0 tx=0).stop 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 shutdown_connections 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f17e8039200 0x7f17e803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 --2- 172.21.15.38:0/1933891287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17fc107510 0x7f17fc19b4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 >> 172.21.15.38:0/1933891287 conn(0x7f17fc102ce0 msgr2=0x7f17fc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 shutdown_connections 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:13.516+0000 7f18008d1640 1 -- 172.21.15.38:0/1933891287 wait complete. 2024-09-19T05:32:14.519 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-09-19T05:32:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:15 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4274015407' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f056da5b640 1 Processor -- start 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f056da5b640 1 -- start start 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f056da5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f056da5b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f056807c7f0 con 0x7f056807de00 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41296/0 (socket says 172.21.15.38:41296) 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.953+0000 7f0566ffd640 1 -- 172.21.15.38:0/4099360840 learned_addr learned my addr 172.21.15.38:0/4099360840 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.954+0000 7f0566ffd640 1 -- 172.21.15.38:0/4099360840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056807c970 con 0x7f056807de00 2024-09-19T05:32:15.713 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.954+0000 7f0566ffd640 1 --2- 172.21.15.38:0/4099360840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0558009820 tx=0x7f055802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1dcc872a49e8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f05667fc640 1 -- 172.21.15.38:0/4099360840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0558037070 con 0x7f056807de00 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f05667fc640 1 -- 172.21.15.38:0/4099360840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0558032920 con 0x7f056807de00 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f05667fc640 1 -- 172.21.15.38:0/4099360840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0558030810 con 0x7f056807de00 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 -- 172.21.15.38:0/4099360840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 msgr2=0x7f056807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 --2- 172.21.15.38:0/4099360840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0558009820 tx=0x7f055802a270 comp rx=0 tx=0).stop 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 -- 172.21.15.38:0/4099360840 shutdown_connections 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 --2- 172.21.15.38:0/4099360840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056807c2b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 -- 172.21.15.38:0/4099360840 >> 172.21.15.38:0/4099360840 conn(0x7f0568102ce0 msgr2=0x7f0568105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 -- 172.21.15.38:0/4099360840 shutdown_connections 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.955+0000 7f056da5b640 1 -- 172.21.15.38:0/4099360840 wait complete. 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f056da5b640 1 Processor -- start 2024-09-19T05:32:15.714 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f056da5b640 1 -- start start 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f056da5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f056da5b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f056810db40 con 0x7f056807de00 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f0566ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41304/0 (socket says 172.21.15.38:41304) 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.956+0000 7f0566ffd640 1 -- 172.21.15.38:0/4274015407 learned_addr learned my addr 172.21.15.38:0/4274015407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f0566ffd640 1 -- 172.21.15.38:0/4274015407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819dc10 con 0x7f056807de00 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f0566ffd640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0558030380 tx=0x7f0558032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0558037040 con 0x7f056807de00 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0558040070 con 0x7f056807de00 2024-09-19T05:32:15.715 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819dea0 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f055803a7b0 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.957+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681a0b90 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.958+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f055803a9e0 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.958+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056807c330 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.958+0000 7f056ca59640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 0x7f055403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.958+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f055806cb10 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.958+0000 7f055ffff640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 0x7f055403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.959+0000 7f055ffff640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 0x7f055403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05500098c0 tx=0x7f0550006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:14.963+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0558044c70 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.141+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f05681a1180 con 0x7f056807de00 2024-09-19T05:32:15.716 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.141+0000 7f056ca59640 1 -- 172.21.15.38:0/4274015407 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f055803ae20 con 0x7f056807de00 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.143+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 msgr2=0x7f055403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.143+0000 7f056da5b640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 0x7f055403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05500098c0 tx=0x7f0550006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.143+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 msgr2=0x7f056819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0558030380 tx=0x7f0558032670 comp rx=0 tx=0).stop 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 shutdown_connections 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0554039260 0x7f055403b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 --2- 172.21.15.38:0/4274015407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f056807de00 0x7f056819d6d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 >> 172.21.15.38:0/4274015407 conn(0x7f0568102ce0 msgr2=0x7f0568103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 shutdown_connections 2024-09-19T05:32:15.717 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:15.144+0000 7f056da5b640 1 -- 172.21.15.38:0/4274015407 wait complete. 2024-09-19T05:32:15.720 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T05:32:15.720 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-19T05:32:15.728 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T05:32:15.728 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-19T05:32:15.728 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-19T05:32:15.728 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout: Password: qqgptz0z09 2024-09-19T05:32:15.730 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:15.731 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config directory 2024-09-19T05:32:16.997 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.165+0000 7feb47eea640 1 Processor -- start 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.165+0000 7feb47eea640 1 -- start start 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.165+0000 7feb47eea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.165+0000 7feb47eea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feb40106180 con 0x7feb401057b0 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.166+0000 7feb45c86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.166+0000 7feb45c86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41308/0 (socket says 172.21.15.38:41308) 2024-09-19T05:32:17.004 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.166+0000 7feb45c86640 1 -- 172.21.15.38:0/83146980 learned_addr learned my addr 172.21.15.38:0/83146980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.166+0000 7feb45c86640 1 -- 172.21.15.38:0/83146980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb40108790 con 0x7feb401057b0 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb45c86640 1 --2- 172.21.15.38:0/83146980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009820 tx=0x7feb3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a805b4d58b0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb44c84640 1 -- 172.21.15.38:0/83146980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb3c037070 con 0x7feb401057b0 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb44c84640 1 -- 172.21.15.38:0/83146980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c032920 con 0x7feb401057b0 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb44c84640 1 -- 172.21.15.38:0/83146980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb3c030810 con 0x7feb401057b0 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb47eea640 1 -- 172.21.15.38:0/83146980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 msgr2=0x7feb40105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb47eea640 1 --2- 172.21.15.38:0/83146980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009820 tx=0x7feb3c02a270 comp rx=0 tx=0).stop 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb47eea640 1 -- 172.21.15.38:0/83146980 shutdown_connections 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb47eea640 1 --2- 172.21.15.38:0/83146980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb40105bb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.167+0000 7feb47eea640 1 -- 172.21.15.38:0/83146980 >> 172.21.15.38:0/83146980 conn(0x7feb40101360 msgr2=0x7feb40103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 -- 172.21.15.38:0/83146980 shutdown_connections 2024-09-19T05:32:17.005 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 -- 172.21.15.38:0/83146980 wait complete. 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 Processor -- start 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 -- start start 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb47eea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feb4010b9c0 con 0x7feb401057b0 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.168+0000 7feb45c86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb45c86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41318/0 (socket says 172.21.15.38:41318) 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb45c86640 1 -- 172.21.15.38:0/2372687967 learned_addr learned my addr 172.21.15.38:0/2372687967 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb45c86640 1 -- 172.21.15.38:0/2372687967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4019ba80 con 0x7feb401057b0 2024-09-19T05:32:17.006 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb45c86640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097f0 tx=0x7feb3c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb3c037040 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c030cc0 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4019bd10 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb3c03a640 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.169+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4019ea00 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.170+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7feb3c03b3f0 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.170+0000 7feb36ffd640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 0x7feb2c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.170+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40105c30 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.170+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb3c06c910 con 0x7feb401057b0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.170+0000 7feb45485640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 0x7feb2c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.171+0000 7feb45485640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 0x7feb2c03b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb300098c0 tx=0x7feb30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:17.007 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.175+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb3c0328c0 con 0x7feb401057b0 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.195+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7feb3c03ed60 con 0x7feb401057b0 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.442+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7feb4019eff0 con 0x7feb401057b0 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.445+0000 7feb36ffd640 1 -- 172.21.15.38:0/2372687967 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7feb3c064110 con 0x7feb401057b0 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 msgr2=0x7feb2c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 0x7feb2c03b720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb300098c0 tx=0x7feb30006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 msgr2=0x7feb4019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0097f0 tx=0x7feb3c0327b0 comp rx=0 tx=0).stop 2024-09-19T05:32:17.008 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 shutdown_connections 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb2c039260 0x7feb2c03b720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 --2- 172.21.15.38:0/2372687967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb401057b0 0x7feb4019b540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 >> 172.21.15.38:0/2372687967 conn(0x7feb40101360 msgr2=0x7feb40101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 shutdown_connections 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-19T05:32:16.448+0000 7feb47eea640 1 -- 172.21.15.38:0/2372687967 wait complete. 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T05:32:17.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T05:32:17.010 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:17.011 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-09-19T05:32:17.045 INFO:tasks.cephadm:Fetching config... 2024-09-19T05:32:17.045 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:17.045 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T05:32:17.063 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T05:32:17.063 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:17.063 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T05:32:17.121 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T05:32:17.121 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:17.121 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T05:32:17.188 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T05:32:17.189 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:17.189 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T05:32:17.244 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T05:32:17.245 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:32:17.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:17 smithi038 ceph-mon[31227]: mgrmap e13: x(active, since 6s) 2024-09-19T05:32:17.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2372687967' entity='client.admin' 2024-09-19T05:32:17.322 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:17.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:32:17.377 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:17.389 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T05:32:17.431 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVrYXFOY7BOFj33M4GennvVvIfnDlQIJS8w6wYNTvRZggv04DeFMERqonFKCG+5l/rkeF4IfqcL4AZUoinpwGs9DRnyVQLtsfr4eWFmzfBbjLjsgi5c1AnYEsGaFseKRCmoCt/AWksEc3Y4YDEDms9f6zVhI4qnriPyBxxC0nPJpNrV+r2+qVqw9YfD0RCl4wgCEAYdvefYvwbcdTWKgjk3oNzc+uwLkRr5CyejdyCKGrXH+2Gm7bA+AbIXCuuZ9082UHyBBEwejmX2vpdbXnaoieGYCTC5EAAEdjCZbyTfKDRl0gJ4JzupuMMuu0OtqCeIh8/swIkDJX9/FxKenp43B6cxtBjNOFtpam+lIUDDVY9xRMJHpndUdc0auetA8Kn/Ib8w0w0JT7D9/rlZAzXOtmqz5oGUDnWgeXIVl2AFRzdX411gTrdOMMsgkY02cA7ytyRKLaun+BbpMES/gIO0/ixRBJ4jrtmS0BvRkblncnP+NCAQL+84ZUy5uVaWwk= ceph-44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:17.443 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T05:32:17.695 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 -- 172.21.15.38:0/525545230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 msgr2=0x7f8994101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 --2- 172.21.15.38:0/525545230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8984009880 tx=0x7f898402a5d0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 -- 172.21.15.38:0/525545230 shutdown_connections 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 --2- 172.21.15.38:0/525545230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 -- 172.21.15.38:0/525545230 >> 172.21.15.38:0/525545230 conn(0x7f89940fcdb0 msgr2=0x7f89940ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 -- 172.21.15.38:0/525545230 shutdown_connections 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.159+0000 7f8999a18640 1 -- 172.21.15.38:0/525545230 wait complete. 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8999a18640 1 Processor -- start 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8999a18640 1 -- start start 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8999a18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8999a18640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8994107710 con 0x7f8994101600 2024-09-19T05:32:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8992ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8992ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39114/0 (socket says 172.21.15.38:39114) 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8992ffd640 1 -- 172.21.15.38:0/710821812 learned_addr learned my addr 172.21.15.38:0/710821812 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8992ffd640 1 -- 172.21.15.38:0/710821812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940763f0 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.160+0000 7f8992ffd640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8984004f60 tx=0x7f89840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f898403f070 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8984004cc0 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89940745e0 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8984035030 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8994074b20 con 0x7f8994101600 2024-09-19T05:32:18.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.161+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8984042070 con 0x7f8994101600 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.162+0000 7f8998a16640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 0x7f898003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.162+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f898406d110 con 0x7f8994101600 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.162+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8994107e90 con 0x7f8994101600 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.162+0000 7f898a7fc640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 0x7f898003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.162+0000 7f898a7fc640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 0x7f898003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f897c0098c0 tx=0x7f897c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.167+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f89840433b0 con 0x7f8994101600 2024-09-19T05:32:18.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.345+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f89941900b0 con 0x7f8994101600 2024-09-19T05:32:18.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.348+0000 7f8998a16640 1 -- 172.21.15.38:0/710821812 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f898402f300 con 0x7f8994101600 2024-09-19T05:32:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 msgr2=0x7f898003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 0x7f898003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f897c0098c0 tx=0x7f897c006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 msgr2=0x7f8994075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8984004f60 tx=0x7f89840048f0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 shutdown_connections 2024-09-19T05:32:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89800392a0 0x7f898003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 --2- 172.21.15.38:0/710821812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8994101600 0x7f8994075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.350+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 >> 172.21.15.38:0/710821812 conn(0x7f89940fcdb0 msgr2=0x7f89940fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.351+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 shutdown_connections 2024-09-19T05:32:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:18.351+0000 7f8999a18640 1 -- 172.21.15.38:0/710821812 wait complete. 2024-09-19T05:32:19.312 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T05:32:19.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T05:32:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/710821812' entity='client.admin' 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.982+0000 7fd1c4964640 1 -- 172.21.15.38:0/829796787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 msgr2=0x7fd1c0100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.982+0000 7fd1c4964640 1 --2- 172.21.15.38:0/829796787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009c20 tx=0x7fd1b002aa90 comp rx=0 tx=0).stop 2024-09-19T05:32:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 -- 172.21.15.38:0/829796787 shutdown_connections 2024-09-19T05:32:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 --2- 172.21.15.38:0/829796787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:19.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 -- 172.21.15.38:0/829796787 >> 172.21.15.38:0/829796787 conn(0x7fd1c00fbcd0 msgr2=0x7fd1c00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 -- 172.21.15.38:0/829796787 shutdown_connections 2024-09-19T05:32:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 -- 172.21.15.38:0/829796787 wait complete. 2024-09-19T05:32:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 Processor -- start 2024-09-19T05:32:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.983+0000 7fd1c4964640 1 -- start start 2024-09-19T05:32:19.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1c4964640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1c4964640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c01065f0 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1be59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1be59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39140/0 (socket says 172.21.15.38:39140) 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1be59c640 1 -- 172.21.15.38:0/1759888081 learned_addr learned my addr 172.21.15.38:0/1759888081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1be59c640 1 -- 172.21.15.38:0/1759888081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c01972f0 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1be59c640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005cc0 tx=0x7fd1b002fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b0047070 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c0197580 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b0003e50 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b0033050 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.984+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c019a230 con 0x7fd1c0100540 2024-09-19T05:32:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.985+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd1b004a070 con 0x7fd1c0100540 2024-09-19T05:32:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.985+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c0106d70 con 0x7fd1c0100540 2024-09-19T05:32:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.985+0000 7fd1af7fe640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 0x7fd1a803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.986+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1b003e030 con 0x7fd1c0100540 2024-09-19T05:32:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.986+0000 7fd1bdd9b640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 0x7fd1a803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.986+0000 7fd1bdd9b640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 0x7fd1a803b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098c0 tx=0x7fd1b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:19.990+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1b00373b0 con 0x7fd1c0100540 2024-09-19T05:32:20.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.167+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd1c002d940 con 0x7fd1a80392a0 2024-09-19T05:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.176+0000 7fd1af7fe640 1 -- 172.21.15.38:0/1759888081 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd1c002d940 con 0x7fd1a80392a0 2024-09-19T05:32:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 msgr2=0x7fd1a803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 0x7fd1a803b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098c0 tx=0x7fd1b4006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 msgr2=0x7fd1c0196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005cc0 tx=0x7fd1b002fc10 comp rx=0 tx=0).stop 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 shutdown_connections 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd1a80392a0 0x7fd1a803b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 --2- 172.21.15.38:0/1759888081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1c0100540 0x7fd1c0196db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 >> 172.21.15.38:0/1759888081 conn(0x7fd1c00fbcd0 msgr2=0x7fd1c0069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 shutdown_connections 2024-09-19T05:32:20.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:20.178+0000 7fd1c4964640 1 -- 172.21.15.38:0/1759888081 wait complete. 2024-09-19T05:32:20.926 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi047 2024-09-19T05:32:20.926 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:32:20.926 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/etc/ceph/ceph.conf 2024-09-19T05:32:20.947 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:32:20.948 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:21.008 INFO:tasks.cephadm:Adding host smithi047 to orchestrator... 2024-09-19T05:32:21.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch host add smithi047 2024-09-19T05:32:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T05:32:21.477 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: Updating smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:32:21.478 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:21.478 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:21.478 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:21.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 -- 172.21.15.38:0/3595006898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 msgr2=0x7f1ff8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:21.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 --2- 172.21.15.38:0/3595006898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8102b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009810 tx=0x7f1fe802a540 comp rx=0 tx=0).stop 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 -- 172.21.15.38:0/3595006898 shutdown_connections 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 --2- 172.21.15.38:0/3595006898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8102b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 -- 172.21.15.38:0/3595006898 >> 172.21.15.38:0/3595006898 conn(0x7f1ff8077310 msgr2=0x7f1ff80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 -- 172.21.15.38:0/3595006898 shutdown_connections 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 -- 172.21.15.38:0/3595006898 wait complete. 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.690+0000 7f1ffd904640 1 Processor -- start 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ffd904640 1 -- start start 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ffd904640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ffd904640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff810bc80 con 0x7f1ff81027b0 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39166/0 (socket says 172.21.15.38:39166) 2024-09-19T05:32:21.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ff6ffd640 1 -- 172.21.15.38:0/884364642 learned_addr learned my addr 172.21.15.38:0/884364642 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ff6ffd640 1 -- 172.21.15.38:0/884364642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8102300 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ff6ffd640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80045e0 tx=0x7f1fe8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.691+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe8030e70 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.692+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe80318e0 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.692+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff8100950 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.692+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe80049c0 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.692+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff8100e10 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.692+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1fe8039040 con 0x7f1ff81027b0 2024-09-19T05:32:21.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.693+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8069f10 con 0x7f1ff81027b0 2024-09-19T05:32:21.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.693+0000 7f1ffc902640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 0x7f1fe403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:21.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.693+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1fe806d680 con 0x7f1ff81027b0 2024-09-19T05:32:21.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.693+0000 7f1feffff640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 0x7f1fe403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:21.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.693+0000 7f1feffff640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 0x7f1fe403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098c0 tx=0x7f1fe0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:21.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.698+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1fe8040030 con 0x7f1ff81027b0 2024-09-19T05:32:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:21.879+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "target": ["mon-mgr", ""]}) -- 0x7f1ff81019b0 con 0x7f1fe40392a0 2024-09-19T05:32:22.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:22 smithi038 ceph-mon[31227]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:22.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:22 smithi038 ceph-mon[31227]: Updating smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:32:23.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:23 smithi038 ceph-mon[31227]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:24 smithi038 ceph-mon[31227]: Deploying cephadm binary to smithi047 2024-09-19T05:32:25.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.839+0000 7f1ffc902640 1 -- 172.21.15.38:0/884364642 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1ff81019b0 con 0x7f1fe40392a0 2024-09-19T05:32:25.842 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi047' with addr '172.21.15.47' 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 msgr2=0x7f1fe403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 0x7f1fe403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098c0 tx=0x7f1fe0006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 msgr2=0x7f1ff8100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80045e0 tx=0x7f1fe8004610 comp rx=0 tx=0).stop 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 shutdown_connections 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1fe40392a0 0x7f1fe403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 --2- 172.21.15.38:0/884364642 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ff81027b0 0x7f1ff8100410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:25.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 >> 172.21.15.38:0/884364642 conn(0x7f1ff8077310 msgr2=0x7f1ff8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:25.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 shutdown_connections 2024-09-19T05:32:25.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:25.842+0000 7f1ffd904640 1 -- 172.21.15.38:0/884364642 wait complete. 2024-09-19T05:32:26.555 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T05:32:26.808 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:27.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:26 smithi038 ceph-mon[31227]: Added host smithi047 2024-09-19T05:32:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:27.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 -- 172.21.15.38:0/3726607554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 msgr2=0x7ff644106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:27.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 --2- 172.21.15.38:0/3726607554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009880 tx=0x7ff62c02a5d0 comp rx=0 tx=0).stop 2024-09-19T05:32:27.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 -- 172.21.15.38:0/3726607554 shutdown_connections 2024-09-19T05:32:27.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 --2- 172.21.15.38:0/3726607554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:27.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 -- 172.21.15.38:0/3726607554 >> 172.21.15.38:0/3726607554 conn(0x7ff6440fc1b0 msgr2=0x7ff6440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:27.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 -- 172.21.15.38:0/3726607554 shutdown_connections 2024-09-19T05:32:27.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.273+0000 7ff64aabb640 1 -- 172.21.15.38:0/3726607554 wait complete. 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff64aabb640 1 Processor -- start 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff64aabb640 1 -- start start 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff64aabb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff64aabb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff64410ab90 con 0x7ff644106560 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff648857640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff648857640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39178/0 (socket says 172.21.15.38:39178) 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff648857640 1 -- 172.21.15.38:0/2674427676 learned_addr learned my addr 172.21.15.38:0/2674427676 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.274+0000 7ff648857640 1 -- 172.21.15.38:0/2674427676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441976b0 con 0x7ff644106560 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff648857640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009230 tx=0x7ff62c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:27.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c03f070 con 0x7ff644106560 2024-09-19T05:32:27.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff644197940 con 0x7ff644106560 2024-09-19T05:32:27.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff62c03bc80 con 0x7ff644106560 2024-09-19T05:32:27.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff62c035030 con 0x7ff644106560 2024-09-19T05:32:27.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.275+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff644197da0 con 0x7ff644106560 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.276+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff62c042070 con 0x7ff644106560 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.276+0000 7ff641ffb640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 0x7ff63003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.276+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff644101bd0 con 0x7ff644106560 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.276+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff62c06d090 con 0x7ff644106560 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.276+0000 7ff643fff640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 0x7ff63003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:27.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.277+0000 7ff643fff640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 0x7ff63003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098c0 tx=0x7ff634006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:27.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.281+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff62c0372d0 con 0x7ff644106560 2024-09-19T05:32:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.461+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff644198540 con 0x7ff6300392a0 2024-09-19T05:32:27.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.463+0000 7ff641ffb640 1 -- 172.21.15.38:0/2674427676 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff644198540 con 0x7ff6300392a0 2024-09-19T05:32:27.464 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:27.464 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}] 2024-09-19T05:32:27.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 msgr2=0x7ff63003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 0x7ff63003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098c0 tx=0x7ff634006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 msgr2=0x7ff644197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009230 tx=0x7ff62c003d40 comp rx=0 tx=0).stop 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 shutdown_connections 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff6300392a0 0x7ff63003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 --2- 172.21.15.38:0/2674427676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff644106560 0x7ff644197170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 >> 172.21.15.38:0/2674427676 conn(0x7ff6440fc1b0 msgr2=0x7ff644069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 shutdown_connections 2024-09-19T05:32:27.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:27.465+0000 7ff64aabb640 1 -- 172.21.15.38:0/2674427676 wait complete. 2024-09-19T05:32:28.062 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-09-19T05:32:28.062 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:32:28.062 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-09-19T05:32:28.082 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:32:28.082 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:28.139 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-09-19T05:32:28.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch host add smithi179 2024-09-19T05:32:28.392 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:28.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:28.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:28 smithi038 ceph-mon[31227]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/2496935352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 msgr2=0x7ff1d4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/2496935352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d4102b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009c10 tx=0x7ff1c402aa10 comp rx=0 tx=0).stop 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/2496935352 shutdown_connections 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/2496935352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d4102b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/2496935352 >> 172.21.15.38:0/2496935352 conn(0x7ff1d4077310 msgr2=0x7ff1d40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/2496935352 shutdown_connections 2024-09-19T05:32:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.796+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/2496935352 wait complete. 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1dbfa9640 1 Processor -- start 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1dbfa9640 1 -- start start 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1dbfa9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1dbfa9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410bc80 con 0x7ff1d4102790 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1d9d45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1d9d45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60002/0 (socket says 172.21.15.38:60002) 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.797+0000 7ff1d9d45640 1 -- 172.21.15.38:0/1271472417 learned_addr learned my addr 172.21.15.38:0/1271472417 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1d9d45640 1 -- 172.21.15.38:0/1271472417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d41022e0 con 0x7ff1d4102790 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1d9d45640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4005dd0 tx=0x7ff1c4030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c4034070 con 0x7ff1d4102790 2024-09-19T05:32:28.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d4100930 con 0x7ff1d4102790 2024-09-19T05:32:28.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c402f8c0 con 0x7ff1d4102790 2024-09-19T05:32:28.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c4004520 con 0x7ff1d4102790 2024-09-19T05:32:28.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.798+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d4100d30 con 0x7ff1d4102790 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.799+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff1c4037020 con 0x7ff1d4102790 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.799+0000 7ff1d2ffd640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 0x7ff1bc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.799+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff1c406d790 con 0x7ff1d4102790 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.799+0000 7ff1d9544640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 0x7ff1bc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.799+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d4069f30 con 0x7ff1d4102790 2024-09-19T05:32:28.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.800+0000 7ff1d9544640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 0x7ff1bc03b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80098c0 tx=0x7ff1c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:28.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.804+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1c4050030 con 0x7ff1d4102790 2024-09-19T05:32:28.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:28.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7ff1d4101920 con 0x7ff1bc0392a0 2024-09-19T05:32:29.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:29 smithi038 ceph-mon[31227]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: Deploying cephadm binary to smithi179 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: Updating smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:32.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:31 smithi038 ceph-mon[31227]: Updating smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:32:32.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.980+0000 7ff1d2ffd640 1 -- 172.21.15.38:0/1271472417 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff1d4101920 con 0x7ff1bc0392a0 2024-09-19T05:32:32.983 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 msgr2=0x7ff1bc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 0x7ff1bc03b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80098c0 tx=0x7ff1c8006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 msgr2=0x7ff1d41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4005dd0 tx=0x7ff1c4030c30 comp rx=0 tx=0).stop 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 shutdown_connections 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff1bc0392a0 0x7ff1bc03b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 --2- 172.21.15.38:0/1271472417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1d4102790 0x7ff1d41003f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 >> 172.21.15.38:0/1271472417 conn(0x7ff1d4077310 msgr2=0x7ff1d4103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 shutdown_connections 2024-09-19T05:32:32.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:32.983+0000 7ff1dbfa9640 1 -- 172.21.15.38:0/1271472417 wait complete. 2024-09-19T05:32:33.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:32 smithi038 ceph-mon[31227]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:33.633 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T05:32:33.883 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:34.238 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:34.239 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:33 smithi038 ceph-mon[31227]: Added host smithi179 2024-09-19T05:32:34.239 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:34.239 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:34.239 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 -- 172.21.15.38:0/3284754785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 msgr2=0x7ffa14107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 --2- 172.21.15.38:0/3284754785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14107a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009880 tx=0x7ffa0802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 -- 172.21.15.38:0/3284754785 shutdown_connections 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 --2- 172.21.15.38:0/3284754785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14107a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 -- 172.21.15.38:0/3284754785 >> 172.21.15.38:0/3284754785 conn(0x7ffa140fd260 msgr2=0x7ffa140ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 -- 172.21.15.38:0/3284754785 shutdown_connections 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.321+0000 7ffa1c233640 1 -- 172.21.15.38:0/3284754785 wait complete. 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa1c233640 1 Processor -- start 2024-09-19T05:32:34.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa1c233640 1 -- start start 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa1c233640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa1c233640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1410bc80 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa19fcf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa19fcf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60026/0 (socket says 172.21.15.38:60026) 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa19fcf640 1 -- 172.21.15.38:0/301488357 learned_addr learned my addr 172.21.15.38:0/301488357 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa19fcf640 1 -- 172.21.15.38:0/301488357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14197660 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.322+0000 7ffa19fcf640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009230 tx=0x7ffa08003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa0803f070 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0803bc80 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141978f0 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa08035030 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1419b9f0 con 0x7ffa14107650 2024-09-19T05:32:34.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.323+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ffa08042070 con 0x7ffa14107650 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.324+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14102cc0 con 0x7ffa14107650 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.324+0000 7ffa06ffd640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 0x7ffa0003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.324+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa0806d1b0 con 0x7ffa14107650 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.324+0000 7ffa197ce640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 0x7ffa0003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.324+0000 7ffa197ce640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 0x7ffa0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa100098c0 tx=0x7ffa10006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:34.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.329+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffa080314c0 con 0x7ffa14107650 2024-09-19T05:32:34.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.511+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa14198480 con 0x7ffa00039300 2024-09-19T05:32:34.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.512+0000 7ffa06ffd640 1 -- 172.21.15.38:0/301488357 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7ffa14198480 con 0x7ffa00039300 2024-09-19T05:32:34.513 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:32:34.513 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-09-19T05:32:34.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 msgr2=0x7ffa0003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 0x7ffa0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa100098c0 tx=0x7ffa10006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 msgr2=0x7ffa14197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009230 tx=0x7ffa08003d40 comp rx=0 tx=0).stop 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 shutdown_connections 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffa00039300 0x7ffa0003b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 --2- 172.21.15.38:0/301488357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa14107650 0x7ffa14197120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 >> 172.21.15.38:0/301488357 conn(0x7ffa140fd260 msgr2=0x7ffa140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 shutdown_connections 2024-09-19T05:32:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:34.514+0000 7ffa1c233640 1 -- 172.21.15.38:0/301488357 wait complete. 2024-09-19T05:32:35.148 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T05:32:35.148 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T05:32:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:35 smithi038 ceph-mon[31227]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:35 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:35.400 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 -- 172.21.15.38:0/2786233863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 msgr2=0x7f53c8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 --2- 172.21.15.38:0/2786233863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c8107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009880 tx=0x7f53c402a5d0 comp rx=0 tx=0).stop 2024-09-19T05:32:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 -- 172.21.15.38:0/2786233863 shutdown_connections 2024-09-19T05:32:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 --2- 172.21.15.38:0/2786233863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c8107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 -- 172.21.15.38:0/2786233863 >> 172.21.15.38:0/2786233863 conn(0x7f53c80fd290 msgr2=0x7f53c80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 -- 172.21.15.38:0/2786233863 shutdown_connections 2024-09-19T05:32:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.803+0000 7f53cf711640 1 -- 172.21.15.38:0/2786233863 wait complete. 2024-09-19T05:32:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cf711640 1 Processor -- start 2024-09-19T05:32:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cf711640 1 -- start start 2024-09-19T05:32:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cf711640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cf711640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f53c810bc90 con 0x7f53c8107660 2024-09-19T05:32:35.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cd4ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cd4ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60040/0 (socket says 172.21.15.38:60040) 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.804+0000 7f53cd4ad640 1 -- 172.21.15.38:0/1065175739 learned_addr learned my addr 172.21.15.38:0/1065175739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53cd4ad640 1 -- 172.21.15.38:0/1065175739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c8199830 con 0x7f53c8107660 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53cd4ad640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009230 tx=0x7f53c4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c403f070 con 0x7f53c8107660 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c403bc80 con 0x7f53c8107660 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c8199ac0 con 0x7f53c8107660 2024-09-19T05:32:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4035030 con 0x7f53c8107660 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.805+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c819dbc0 con 0x7f53c8107660 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.806+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f53c4042070 con 0x7f53c8107660 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.806+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c8102cd0 con 0x7f53c8107660 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.806+0000 7f53be7fc640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 0x7f53b403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.806+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f53c406d140 con 0x7f53c8107660 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.806+0000 7f53cccac640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 0x7f53b403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:35.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.807+0000 7f53cccac640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 0x7f53b403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098c0 tx=0x7f53b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:35.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.811+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f53c404c350 con 0x7f53c8107660 2024-09-19T05:32:35.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:35.993+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f53c8107ac0 con 0x7f53c8107660 2024-09-19T05:32:36.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.324+0000 7f53be7fc640 1 -- 172.21.15.38:0/1065175739 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f53c40314c0 con 0x7f53c8107660 2024-09-19T05:32:36.327 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 msgr2=0x7f53b403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 0x7f53b403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098c0 tx=0x7f53b8006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 msgr2=0x7f53c81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009230 tx=0x7f53c4003d40 comp rx=0 tx=0).stop 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 shutdown_connections 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f53b40392a0 0x7f53b403b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 --2- 172.21.15.38:0/1065175739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c8107660 0x7f53c81992f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 >> 172.21.15.38:0/1065175739 conn(0x7f53c80fd290 msgr2=0x7f53c8075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 shutdown_connections 2024-09-19T05:32:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:36.327+0000 7f53cf711640 1 -- 172.21.15.38:0/1065175739 wait complete. 2024-09-19T05:32:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:36 smithi038 ceph-mon[31227]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:32:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:36 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1065175739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T05:32:37.059 INFO:tasks.cephadm:Adding mon.a on smithi038 2024-09-19T05:32:37.059 INFO:tasks.cephadm:Adding mon.b on smithi047 2024-09-19T05:32:37.059 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-09-19T05:32:37.059 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c' 2024-09-19T05:32:37.308 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T05:32:37.354 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1065175739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:37.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 -- 172.21.15.179:0/146323786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 msgr2=0x7f10e81044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:37.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 --2- 172.21.15.179:0/146323786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e81044f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009c20 tx=0x7f10d802aa90 comp rx=0 tx=0).stop 2024-09-19T05:32:37.944 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 -- 172.21.15.179:0/146323786 shutdown_connections 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 --2- 172.21.15.179:0/146323786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e81044f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 -- 172.21.15.179:0/146323786 >> 172.21.15.179:0/146323786 conn(0x7f10e80fbca0 msgr2=0x7f10e80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 -- 172.21.15.179:0/146323786 shutdown_connections 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.888+0000 7f10ed912640 1 -- 172.21.15.179:0/146323786 wait complete. 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10ed912640 1 Processor -- start 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10ed912640 1 -- start start 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10ed912640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10ed912640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8107710 con 0x7f10e8102100 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.179:46514/0 (socket says 172.21.15.179:46514) 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10e6ffd640 1 -- 172.21.15.179:0/3575300850 learned_addr learned my addr 172.21.15.179:0/3575300850 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.889+0000 7f10e6ffd640 1 -- 172.21.15.179:0/3575300850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e8101ec0 con 0x7f10e8102100 2024-09-19T05:32:37.945 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10e6ffd640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d802e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d8047070 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81000b0 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8003e50 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d8033050 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.890+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81005f0 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.891+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f10d804a070 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.891+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8107e90 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.891+0000 7f10ec910640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 0x7f10d403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.891+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f10d803e030 con 0x7f10e8102100 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.891+0000 7f10dffff640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 0x7f10d403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.892+0000 7f10dffff640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 0x7f10d403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098c0 tx=0x7f10d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:37.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:37.896+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10d804ade0 con 0x7f10e8102100 2024-09-19T05:32:38.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.082+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) -- 0x7f10e8190290 con 0x7f10d40392a0 2024-09-19T05:32:38.088 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.087+0000 7f10ec910640 1 -- 172.21.15.179:0/3575300850 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f10e8190290 con 0x7f10d40392a0 2024-09-19T05:32:38.089 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 msgr2=0x7f10d403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 0x7f10d403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098c0 tx=0x7f10d0006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 msgr2=0x7f10e8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d802e3c0 comp rx=0 tx=0).stop 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 shutdown_connections 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f10d40392a0 0x7f10d403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 --2- 172.21.15.179:0/3575300850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10e8102100 0x7f10e8101980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 >> 172.21.15.179:0/3575300850 conn(0x7f10e80fbca0 msgr2=0x7f10e80fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:38.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 shutdown_connections 2024-09-19T05:32:38.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:38.089+0000 7f10ed912640 1 -- 172.21.15.179:0/3575300850 wait complete. 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: Updating smithi179:/etc/ceph/ceph.conf 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:38.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:38.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:32:38.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:38.800 DEBUG:teuthology.orchestra.run.smithi047:mon.b> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T05:32:38.802 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T05:32:38.805 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:32:38.805 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:32:39.171 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c;count:3 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T05:32:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:39 smithi038 ceph-mon[31227]: Reconfiguring daemon mon.a on smithi038 2024-09-19T05:32:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 -- 172.21.15.179:0/2152014924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 msgr2=0x7f9524106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 --2- 172.21.15.179:0/2152014924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9520009810 tx=0x7f952002a540 comp rx=0 tx=0).stop 2024-09-19T05:32:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 -- 172.21.15.179:0/2152014924 shutdown_connections 2024-09-19T05:32:39.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 --2- 172.21.15.179:0/2152014924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 -- 172.21.15.179:0/2152014924 >> 172.21.15.179:0/2152014924 conn(0x7f9524077310 msgr2=0x7f95240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.629+0000 7f952ba7b640 1 -- 172.21.15.179:0/2152014924 shutdown_connections 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f952ba7b640 1 -- 172.21.15.179:0/2152014924 wait complete. 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f952ba7b640 1 Processor -- start 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f952ba7b640 1 -- start start 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f952ba7b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f952ba7b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f952410ab70 con 0x7f9524106540 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f9529817640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.630+0000 7f9529817640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.179:46538/0 (socket says 172.21.15.179:46538) 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f9529817640 1 -- 172.21.15.179:0/3759651126 learned_addr learned my addr 172.21.15.179:0/3759651126 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f9529817640 1 -- 172.21.15.179:0/3759651126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95241965a0 con 0x7f9524106540 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f9529817640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95200045e0 tx=0x7f9520004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:39.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9520030e70 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95200318e0 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9524196830 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95200049c0 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.631+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9524196c90 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.632+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9520039040 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.632+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9524101bb0 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.632+0000 7f951affd640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 0x7f951003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.632+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f952006e660 con 0x7f9524106540 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.633+0000 7f9529016640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 0x7f951003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.633+0000 7f9529016640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 0x7f951003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95140098c0 tx=0x7f9514006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:39.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.637+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9520040030 con 0x7f9524106540 2024-09-19T05:32:39.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.896+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9524197450 con 0x7f9524106540 2024-09-19T05:32:39.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.897+0000 7f951affd640 1 -- 172.21.15.179:0/3759651126 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9524197450 con 0x7f9524106540 2024-09-19T05:32:39.899 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:32:39.899 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","modified":"2024-09-19T05:31:20.164217Z","created":"2024-09-19T05:31:20.164217Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T05:32:39.900 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-09-19T05:32:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.899+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 msgr2=0x7f951003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:39.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.899+0000 7f952ba7b640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 0x7f951003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95140098c0 tx=0x7f9514006db0 comp rx=0 tx=0).stop 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.899+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 msgr2=0x7f9524196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.899+0000 7f952ba7b640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95200045e0 tx=0x7f9520004610 comp rx=0 tx=0).stop 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 shutdown_connections 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f95100392a0 0x7f951003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 --2- 172.21.15.179:0/3759651126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9524106540 0x7f9524196060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 >> 172.21.15.179:0/3759651126 conn(0x7f9524077310 msgr2=0x7f95240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 shutdown_connections 2024-09-19T05:32:39.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:39.900+0000 7f952ba7b640 1 -- 172.21.15.179:0/3759651126 wait complete. 2024-09-19T05:32:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: Deploying daemon mon.c on smithi179 2024-09-19T05:32:40.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:40 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/3759651126' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:42.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:42 smithi038 ceph-mon[31227]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:43.581 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:32:43.581 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:32:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:43 smithi179 systemd[1]: Started Ceph mon.c for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:32:43.827 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:32:48.652 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- 172.21.15.179:0/1497214922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb440003530 msgr2=0x7fb4400059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 --2- 172.21.15.179:0/1497214922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb440003530 0x7fb4400059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb4541016a0 tx=0x7fb44402c490 comp rx=0 tx=0).stop 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- 172.21.15.179:0/1497214922 shutdown_connections 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 --2- 172.21.15.179:0/1497214922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb440003530 0x7fb4400059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- 172.21.15.179:0/1497214922 >> 172.21.15.179:0/1497214922 conn(0x7fb4540fc220 msgr2=0x7fb4540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- 172.21.15.179:0/1497214922 shutdown_connections 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- 172.21.15.179:0/1497214922 wait complete. 2024-09-19T05:32:48.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 Processor -- start 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- start start 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 0x7fb454198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb454108670 con 0x7fb454198600 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb459942640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541084f0 con 0x7fb440003530 2024-09-19T05:32:48.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.650+0000 7fb452ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35542/0 (socket says 172.21.15.179:35542) 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 -- 172.21.15.179:0/2194322203 learned_addr learned my addr 172.21.15.179:0/2194322203 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb4527fc640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 0x7fb454198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 -- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 msgr2=0x7fb4541980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 -- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 msgr2=0x7fb4541980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb452ffd640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb4527fc640 1 -- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 msgr2=0x7fb4541980c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb4527fc640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb4527fc640 1 -- 172.21.15.179:0/2194322203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a1130 con 0x7fb454198600 2024-09-19T05:32:48.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb4527fc640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 0x7fb454198a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00b3d0 tx=0x7fb43c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.651+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb43c011630 con 0x7fb454198600 2024-09-19T05:32:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.652+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb43c011cb0 con 0x7fb454198600 2024-09-19T05:32:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.652+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a1300 con 0x7fb454198600 2024-09-19T05:32:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.652+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb43c010df0 con 0x7fb454198600 2024-09-19T05:32:48.657 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.652+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a1890 con 0x7fb454198600 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.653+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb43c018020 con 0x7fb454198600 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.653+0000 7fb458940640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 0x7fb43403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.653+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb454101ca0 con 0x7fb454198600 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.653+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb43c04bed0 con 0x7fb454198600 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.653+0000 7fb452ffd640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 0x7fb43403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:48.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.654+0000 7fb452ffd640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 0x7fb43403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb444012c10 tx=0x7fb4440323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:48.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.659+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb43c01f020 con 0x7fb454198600 2024-09-19T05:32:48.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: mon.a calling monitor election 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: mon.c calling monitor election 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: monmap epoch 2 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: last_changed 2024-09-19T05:32:43.453301+0000 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:48.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: election_strategy: 1 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: fsmap 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: mgrmap e13: x(active, since 38s) 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: overall HEALTH_OK 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:48.898 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:48.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.894+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb45418fe50 con 0x7fb454198600 2024-09-19T05:32:48.899 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:48.899 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.899 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: mon.a calling monitor election 2024-09-19T05:32:48.899 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: mon.c calling monitor election 2024-09-19T05:32:48.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.901 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.901 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:48.901 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:48.901 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:48.901 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: monmap epoch 2 2024-09-19T05:32:48.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:48.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: last_changed 2024-09-19T05:32:43.453301+0000 2024-09-19T05:32:48.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:48.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: min_mon_release 19 (squid) 2024-09-19T05:32:48.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: election_strategy: 1 2024-09-19T05:32:48.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:48.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:48.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: fsmap 2024-09-19T05:32:48.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: mgrmap e13: x(active, since 38s) 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: overall HEALTH_OK 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:48.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:48.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:48.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.895+0000 7fb458940640 1 -- 172.21.15.179:0/2194322203 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb43c018310 con 0x7fb454198600 2024-09-19T05:32:48.906 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:32:48.906 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","modified":"2024-09-19T05:32:43.453301Z","created":"2024-09-19T05:31:20.164217Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:32:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-09-19T05:32:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 msgr2=0x7fb43403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:48.907 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 0x7fb43403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb444012c10 tx=0x7fb4440323a0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 msgr2=0x7fb454198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 0x7fb454198a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00b3d0 tx=0x7fb43c00b890 comp rx=0 tx=0).stop 2024-09-19T05:32:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 shutdown_connections 2024-09-19T05:32:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb434039090 0x7fb43403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb454198600 0x7fb454198a60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 --2- 172.21.15.179:0/2194322203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb440003530 0x7fb4541980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 >> 172.21.15.179:0/2194322203 conn(0x7fb4540fc220 msgr2=0x7fb4540fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 shutdown_connections 2024-09-19T05:32:48.909 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:48.897+0000 7fb459942640 1 -- 172.21.15.179:0/2194322203 wait complete. 2024-09-19T05:32:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:49 smithi038 ceph-mon[31227]: Deploying daemon mon.b on smithi047 2024-09-19T05:32:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:49 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/2194322203' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:49 smithi179 ceph-mon[35781]: Deploying daemon mon.b on smithi047 2024-09-19T05:32:49.970 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:49 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/2194322203' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:49.970 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:50.466 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:32:50.467 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:32:50.719 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:32:50.840 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:50 smithi179 ceph-mon[35781]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:50.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:50 smithi038 ceph-mon[31227]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 -- 172.21.15.179:0/2841489595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 msgr2=0x7f34f4107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/2841489595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4107ac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009880 tx=0x7f34e402a5d0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 -- 172.21.15.179:0/2841489595 shutdown_connections 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/2841489595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4107ac0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 -- 172.21.15.179:0/2841489595 >> 172.21.15.179:0/2841489595 conn(0x7f34f40fb9a0 msgr2=0x7f34f40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 -- 172.21.15.179:0/2841489595 shutdown_connections 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 -- 172.21.15.179:0/2841489595 wait complete. 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.143+0000 7f34fbdb2640 1 Processor -- start 2024-09-19T05:32:51.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34fbdb2640 1 -- start start 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34fbdb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34fbdb2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34fbdb2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f34f410bd50 con 0x7f34f41076e0 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34fbdb2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34f410bbd0 con 0x7f34f41978d0 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35546/0 (socket says 172.21.15.179:35546) 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 -- 172.21.15.179:0/1357533802 learned_addr learned my addr 172.21.15.179:0/1357533802 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 -- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 msgr2=0x7f34f419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 -- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 msgr2=0x7f34f419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f934d640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:51.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f9b4e640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f9b4e640 1 -- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 msgr2=0x7f34f419bc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f9b4e640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.144+0000 7f34f9b4e640 1 -- 172.21.15.179:0/1357533802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f4197ef0 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34f9b4e640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4197390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005ab0 tx=0x7f34e4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34e4035070 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f419c1a0 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e4032e40 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34e403bc10 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.145+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f419c700 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.146+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f4102dd0 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.146+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f34e4042030 con 0x7f34f41076e0 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.146+0000 7f34f2ffd640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 0x7f34dc03b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:51.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.146+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f34e4040070 con 0x7f34f41076e0 2024-09-19T05:32:51.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.146+0000 7f34f934d640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 0x7f34dc03b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:51.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.147+0000 7f34f934d640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 0x7f34dc03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009400 tx=0x7f34e8007040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:51.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.151+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34e404a260 con 0x7f34f41076e0 2024-09-19T05:32:51.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.381+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34f40008d0 con 0x7f34f41076e0 2024-09-19T05:32:51.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.381+0000 7f34f2ffd640 1 -- 172.21.15.179:0/1357533802 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f34e403bdb0 con 0x7f34f41076e0 2024-09-19T05:32:51.383 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:32:51.384 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","modified":"2024-09-19T05:32:43.453301Z","created":"2024-09-19T05:31:20.164217Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:32:51.384 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-09-19T05:32:51.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 msgr2=0x7f34dc03b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:51.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 0x7f34dc03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009400 tx=0x7f34e8007040 comp rx=0 tx=0).stop 2024-09-19T05:32:51.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 msgr2=0x7f34f4197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:51.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4197390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005ab0 tx=0x7f34e4032c50 comp rx=0 tx=0).stop 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 shutdown_connections 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f34dc039040 0x7f34dc03b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34f41978d0 0x7f34f419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 --2- 172.21.15.179:0/1357533802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34f41076e0 0x7f34f4197390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 >> 172.21.15.179:0/1357533802 conn(0x7f34f40fb9a0 msgr2=0x7f34f40fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.383+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 shutdown_connections 2024-09-19T05:32:51.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:51.384+0000 7f34fbdb2640 1 -- 172.21.15.179:0/1357533802 wait complete. 2024-09-19T05:32:51.849 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:51 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/1357533802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:51 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/1357533802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:52.133 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 podman[35309]: 2024-09-19 05:32:52.132141552 +0000 UTC m=+0.269737764 container init caf1f61d59912133bf96e5b81a0dd0fed29307c54733543a07ea4c80cd7376df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T05:32:52.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 podman[35309]: 2024-09-19 05:32:52.135505743 +0000 UTC m=+0.273101945 container start caf1f61d59912133bf96e5b81a0dd0fed29307c54733543a07ea4c80cd7376df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T05:32:52.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pidfile_write: ignore empty --pid-file 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: load: jerasure load: lrc 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: RocksDB version: 7.9.2 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Git sha 0 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: DB SUMMARY 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: DB Session ID: 18XA5E4P1JQC21TD8GZS 2024-09-19T05:32:52.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: CURRENT file: CURRENT 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.error_if_exists: 0 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.create_if_missing: 0 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.paranoid_checks: 1 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.env: 0x55eb2be7f940 2024-09-19T05:32:52.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.info_log: 0x55eb2d14f6e0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.statistics: (nil) 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.use_fsync: 0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_log_file_size: 0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_fallocate: 1 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T05:32:52.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.use_direct_reads: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.db_log_dir: 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.wal_dir: 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T05:32:52.480 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.write_buffer_manager: 0x55eb2d15edc0 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T05:32:52.481 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.unordered_write: 0 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.row_cache: None 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.wal_filter: None 2024-09-19T05:32:52.482 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T05:32:52.483 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T05:32:52.483 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.two_write_queues: 0 2024-09-19T05:32:52.483 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T05:32:52.483 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.wal_compression: 0 2024-09-19T05:32:52.483 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.atomic_flush: 0 2024-09-19T05:32:52.484 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T05:32:52.484 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T05:32:52.484 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T05:32:52.486 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.log_readahead_size: 0 2024-09-19T05:32:52.486 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T05:32:52.486 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T05:32:52.486 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T05:32:52.487 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T05:32:52.487 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T05:32:52.487 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T05:32:52.537 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_background_jobs: 2 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_background_compactions: -1 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_subcompactions: 1 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T05:32:52.538 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_open_files: -1 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_background_flushes: -1 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Compression algorithms supported: 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kZSTD supported: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kXpressCompression supported: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kBZip2Compression supported: 0 2024-09-19T05:32:52.539 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kLZ4Compression supported: 1 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kZlibCompression supported: 1 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: kSnappyCompression supported: 1 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.merge_operator: 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_filter: None 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T05:32:52.540 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eb2d14f320) 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: index_type: 0 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: data_block_index_type: 0 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: index_shortening: 1 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: checksum: 4 2024-09-19T05:32:52.541 INFO:journalctl@ceph.mon.b.smithi047.stdout: no_block_cache: 0 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_cache: 0x55eb2d1431f0 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_cache_options: 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: capacity : 536870912 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: num_shard_bits : 4 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: strict_capacity_limit : 0 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_cache_compressed: (nil) 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: persistent_cache: (nil) 2024-09-19T05:32:52.542 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_size: 4096 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_size_deviation: 10 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_restart_interval: 16 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: index_block_restart_interval: 1 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: metadata_block_size: 4096 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: partition_filters: 0 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: use_delta_encoding: 1 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: filter_policy: bloomfilter 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: whole_key_filtering: 1 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: verify_compression: 0 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-09-19T05:32:52.543 INFO:journalctl@ceph.mon.b.smithi047.stdout: format_version: 5 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: enable_index_compression: 1 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: block_align: 0 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: max_auto_readahead_size: 262144 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: prepopulate_block_cache: 0 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: initial_auto_readahead_size: 8192 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression: NoCompression 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T05:32:52.544 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.num_levels: 7 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:32:52.545 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T05:32:52.546 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T05:32:52.547 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T05:32:52.548 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T05:32:52.549 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.inplace_update_support: 0 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T05:32:52.550 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.bloom_locality: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.max_successive_merges: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.ttl: 2592000 2024-09-19T05:32:52.551 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enable_blob_files: false 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.min_blob_size: 0 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T05:32:52.552 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f84ba9fd-3ef6-458d-a51f-54ab6102a569 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723972162123, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723972162526, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726723972, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f84ba9fd-3ef6-458d-a51f-54ab6102a569", "db_session_id": "18XA5E4P1JQC21TD8GZS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726723972162620, "job": 1, "event": "recovery_finished"} 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T05:32:52.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eb2d170e00 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: DB pointer 0x55eb2d26c000 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: ** DB Stats ** 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T05:32:52.554 INFO:journalctl@ceph.mon.b.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: ** Compaction Stats [default] ** 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T05:32:52.555 INFO:journalctl@ceph.mon.b.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T05:32:52.556 INFO:journalctl@ceph.mon.b.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: Block cache BinnedLRUCache@0x55eb2d1431f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T05:32:52.557 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: using public_addr v2:172.21.15.47:0/0 -> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: starting mon.b rank -1 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(???) e0 preinit fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: e1 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: btime 2024-09-19T05:31:23:687807+0000 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: legacy client fscid: -1 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout: No filesystems configured 2024-09-19T05:32:52.558 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:32:52.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1065175739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1065175739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.560 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Updating smithi179:/etc/ceph/ceph.conf 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.561 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:32:52.562 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c;count:3 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Reconfiguring daemon mon.a on smithi038 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:32:52.563 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Deploying daemon mon.c on smithi179 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/3759651126' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.a calling monitor election 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.564 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.c calling monitor election 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.565 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: monmap epoch 2 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: last_changed 2024-09-19T05:32:43.453301+0000 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: min_mon_release 19 (squid) 2024-09-19T05:32:52.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: election_strategy: 1 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: fsmap 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mgrmap e13: x(active, since 38s) 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: overall HEALTH_OK 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:32:52.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: Deploying daemon mon.b on smithi047 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/2194322203' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/1357533802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 ceph-mon[35327]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 bash[35309]: caf1f61d59912133bf96e5b81a0dd0fed29307c54733543a07ea4c80cd7376df 2024-09-19T05:32:52.570 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:52 smithi047 systemd[1]: Started Ceph mon.b for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:32:52.971 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T05:32:52.972 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T05:32:53.220 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/3336846081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 msgr2=0x7f1e980059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/3336846081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1e980059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1eac06a240 tx=0x7f1ea802c3f0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/3336846081 shutdown_connections 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/3336846081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1e980059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/3336846081 >> 172.21.15.179:0/3336846081 conn(0x7f1eac0fc280 msgr2=0x7f1eac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/3336846081 shutdown_connections 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.235+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/3336846081 wait complete. 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 Processor -- start 2024-09-19T05:32:57.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 -- start start 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac1097a0 con 0x7f1e98003530 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac109620 con 0x7f1eac1a3580 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb3f5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac109920 con 0x7f1eac1009e0 2024-09-19T05:32:57.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb14f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb14f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50676/0 (socket says 172.21.15.179:50676) 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.179:33072/0 (socket says 172.21.15.179:33072) 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.236+0000 7f1eb14f5640 1 -- 172.21.15.179:0/272338846 learned_addr learned my addr 172.21.15.179:0/272338846 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb24f7640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb14f5640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 msgr2=0x7f1eac100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb14f5640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 msgr2=0x7f1eac100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb14f5640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb14f5640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb24f7640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 msgr2=0x7f1eac1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb24f7640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 msgr2=0x7f1eac1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb24f7640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:32:57.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb24f7640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 msgr2=0x7f1eac100e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 msgr2=0x7f1eac1a5970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 -- 172.21.15.179:0/272338846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a6040 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1eb1cf6640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8005f10 tx=0x7f1ea80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.237+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8012450 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.238+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea8042070 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.238+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a62d0 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.238+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea803d8f0 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.238+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a6830 con 0x7f1e98003530 2024-09-19T05:32:57.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.239+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1ea803da90 con 0x7f1e98003530 2024-09-19T05:32:57.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.239+0000 7f1ea2ffd640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 0x7f1e9003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:57.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.239+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ea806eb30 con 0x7f1e98003530 2024-09-19T05:32:57.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.239+0000 7f1eb14f5640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 0x7f1e9003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:57.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.239+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac10a8d0 con 0x7f1e98003530 2024-09-19T05:32:57.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.240+0000 7f1eb14f5640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 0x7f1e9003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009100 tx=0x7f1e9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:57.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.244+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ea8045c20 con 0x7f1e98003530 2024-09-19T05:32:57.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.490+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1eac1a6a90 con 0x7f1e98003530 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: mon.a calling monitor election 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: mon.c calling monitor election 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:57.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.495 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: monmap epoch 3 2024-09-19T05:32:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:32:57.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:57.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: min_mon_release 19 (squid) 2024-09-19T05:32:57.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: election_strategy: 1 2024-09-19T05:32:57.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:57.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:57.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: fsmap 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: mgrmap e13: x(active, since 47s) 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: overall HEALTH_OK 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.501 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.491+0000 7f1ea2ffd640 1 -- 172.21.15.179:0/272338846 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f1ea8034de0 con 0x7f1e98003530 2024-09-19T05:32:57.501 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:32:57.502 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","modified":"2024-09-19T05:32:52.188134Z","created":"2024-09-19T05:31:20.164217Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T05:32:57.502 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-09-19T05:32:57.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.493+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 msgr2=0x7f1e9003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:57.502 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.493+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 0x7f1e9003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009100 tx=0x7f1e9c008040 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.493+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 msgr2=0x7f1eac1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.493+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8005f10 tx=0x7f1ea80048d0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 shutdown_connections 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e90039730 0x7f1e9003bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eac1a3580 0x7f1eac1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1eac1009e0 0x7f1eac100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 --2- 172.21.15.179:0/272338846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e98003530 0x7f1eac1004a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 >> 172.21.15.179:0/272338846 conn(0x7f1eac0fc280 msgr2=0x7f1eac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 shutdown_connections 2024-09-19T05:32:57.503 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:32:57.494+0000 7f1eb3f5a640 1 -- 172.21.15.179:0/272338846 wait complete. 2024-09-19T05:32:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: mon.a calling monitor election 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: mon.c calling monitor election 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: monmap epoch 3 2024-09-19T05:32:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: election_strategy: 1 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: fsmap 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: mgrmap e13: x(active, since 47s) 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: overall HEALTH_OK 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:57.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:58.102 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T05:32:58.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T05:32:58.370 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:32:58.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:58 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:58.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:58 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/272338846' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:58.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:58 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:58.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:58 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:58.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:58 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/272338846' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:58.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:58 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 -- 172.21.15.38:0/3758375561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc102820 msgr2=0x7f63cc102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/3758375561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc102820 0x7f63cc102c00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009880 tx=0x7f63bc02a670 comp rx=0 tx=0).stop 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 -- 172.21.15.38:0/3758375561 shutdown_connections 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/3758375561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc102820 0x7f63cc102c00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 -- 172.21.15.38:0/3758375561 >> 172.21.15.38:0/3758375561 conn(0x7f63cc0fc220 msgr2=0x7f63cc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 -- 172.21.15.38:0/3758375561 shutdown_connections 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.731+0000 7f63d1fe5640 1 -- 172.21.15.38:0/3758375561 wait complete. 2024-09-19T05:32:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.732+0000 7f63d1fe5640 1 Processor -- start 2024-09-19T05:32:58.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.732+0000 7f63d1fe5640 1 -- start start 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 0x7f63cc100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10bd20 con 0x7f63cc1009c0 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10bba0 con 0x7f63cc1a3750 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63d1fe5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc10bea0 con 0x7f63cc102820 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:58.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 0x7f63cc100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46192/0 (socket says 172.21.15.38:46192) 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 0x7f63cc100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:42440/0 (socket says 172.21.15.38:42440) 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 -- 172.21.15.38:0/573605531 learned_addr learned my addr 172.21.15.38:0/573605531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cbfff640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 msgr2=0x7f63cc100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 0x7f63cc100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cbfff640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 msgr2=0x7f63cc1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cbfff640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 msgr2=0x7f63cc1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 msgr2=0x7f63cc1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:58.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cbfff640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63cbfff640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.733+0000 7f63caffd640 1 -- 172.21.15.38:0/573605531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc1a6210 con 0x7f63cc1009c0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63caffd640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b520 tx=0x7f63c000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004230 con 0x7f63cc1009c0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c00043d0 con 0x7f63cc1009c0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc1a6500 con 0x7f63cc1009c0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0003d60 con 0x7f63cc1009c0 2024-09-19T05:32:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.734+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1a6a90 con 0x7f63cc1009c0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.735+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f63c000e070 con 0x7f63cc1009c0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.735+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63cc102c80 con 0x7f63cc1009c0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.735+0000 7f63c8ff9640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 0x7f63b403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.735+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63c004bbf0 con 0x7f63cc1009c0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.735+0000 7f63cb7fe640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 0x7f63b403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.736+0000 7f63cb7fe640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 0x7f63b403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009230 tx=0x7f63bc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:32:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.740+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63c0014020 con 0x7f63cc1009c0 2024-09-19T05:32:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.918+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f63cc1a6db0 con 0x7f63cc1009c0 2024-09-19T05:32:58.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.918+0000 7f63c8ff9640 1 -- 172.21.15.38:0/573605531 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f63c0020350 con 0x7f63cc1009c0 2024-09-19T05:32:58.921 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:58.921 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-09-19T05:32:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:58.921 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-09-19T05:32:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 msgr2=0x7f63b403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 0x7f63b403bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009230 tx=0x7f63bc005a40 comp rx=0 tx=0).stop 2024-09-19T05:32:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 msgr2=0x7f63cc100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:32:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b520 tx=0x7f63c000b9e0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 shutdown_connections 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63b4039730 0x7f63b403bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63cc1a3750 0x7f63cc1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63cc1009c0 0x7f63cc100e20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.920+0000 7f63d1fe5640 1 --2- 172.21.15.38:0/573605531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63cc102820 0x7f63cc100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.921+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 >> 172.21.15.38:0/573605531 conn(0x7f63cc0fc220 msgr2=0x7f63cc1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.921+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 shutdown_connections 2024-09-19T05:32:58.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:32:58.921+0000 7f63d1fe5640 1 -- 172.21.15.38:0/573605531 wait complete. 2024-09-19T05:32:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:32:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.a calling monitor election 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.c calling monitor election 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: monmap epoch 3 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: min_mon_release 19 (squid) 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: election_strategy: 1 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: fsmap 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mgrmap e13: x(active, since 47s) 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: overall HEALTH_OK 2024-09-19T05:32:59.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/272338846' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T05:32:59.479 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:32:59.680 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T05:32:59.680 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:59.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:32:59.708 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:32:59.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:59.778 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:32:59.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:32:59.809 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:32:59.809 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:59.875 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:32:59.876 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T05:32:59.908 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:32:59.908 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:32:59.974 INFO:tasks.cephadm:Adding mgr.x on smithi038 2024-09-19T05:32:59.974 INFO:tasks.cephadm:Adding mgr.y on smithi047 2024-09-19T05:32:59.975 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-09-19T05:32:59.975 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi038=x;smithi047=y;smithi179=z' 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mon.b calling monitor election 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi179:/etc/ceph/ceph.conf 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: Updating smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mon.b calling monitor election 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mon.c calling monitor election 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mon.a calling monitor election 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: monmap epoch 3 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: min_mon_release 19 (squid) 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: election_strategy: 1 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:33:00.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: fsmap 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: mgrmap e13: x(active, since 49s) 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: overall HEALTH_OK 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:00.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:00.048 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:32:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mon.b calling monitor election 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi179:/etc/ceph/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: Updating smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mon.b calling monitor election 2024-09-19T05:33:00.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mon.c calling monitor election 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mon.a calling monitor election 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: monmap epoch 3 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: min_mon_release 19 (squid) 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: election_strategy: 1 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: fsmap 2024-09-19T05:33:00.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: mgrmap e13: x(active, since 49s) 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: overall HEALTH_OK 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:00.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:32:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.b calling monitor election 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi179:/etc/ceph/ceph.conf 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: Updating smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.b calling monitor election 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.c calling monitor election 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.a calling monitor election 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: monmap epoch 3 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: last_changed 2024-09-19T05:32:52.188134+0000 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: created 2024-09-19T05:31:20.164217+0000 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: min_mon_release 19 (squid) 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: election_strategy: 1 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-19T05:33:00.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: fsmap 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: mgrmap e13: x(active, since 49s) 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: overall HEALTH_OK 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:00.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:00.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:00.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:32:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:00.251 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- 172.21.15.179:0/2552554207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a81016e0 msgr2=0x7f44a8101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 --2- 172.21.15.179:0/2552554207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a81016e0 0x7f44a8101ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4498009880 tx=0x7f449802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- 172.21.15.179:0/2552554207 shutdown_connections 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 --2- 172.21.15.179:0/2552554207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a81016e0 0x7f44a8101ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- 172.21.15.179:0/2552554207 >> 172.21.15.179:0/2552554207 conn(0x7f44a80fce90 msgr2=0x7f44a80ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- 172.21.15.179:0/2552554207 shutdown_connections 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- 172.21.15.179:0/2552554207 wait complete. 2024-09-19T05:33:00.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 Processor -- start 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.696+0000 7f44af454640 1 -- start start 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a81974f0 0x7f44a819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a8197970 0x7f44a819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81077f0 con 0x7f44a8197970 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8107670 con 0x7f44a81974f0 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44af454640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8107970 con 0x7f44a81016e0 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50678/0 (socket says 172.21.15.179:50678) 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 -- 172.21.15.179:0/3723106524 learned_addr learned my addr 172.21.15.179:0/3723106524 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad9f1640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a8197970 0x7f44a819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ac9ef640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a81974f0 0x7f44a819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:00.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 -- 172.21.15.179:0/3723106524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a81974f0 msgr2=0x7f44a819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a81974f0 0x7f44a819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 -- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a8197970 msgr2=0x7f44a819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a8197970 0x7f44a819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 -- 172.21.15.179:0/3723106524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a819f3b0 con 0x7f44a81016e0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f44ad1f0640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4498005ab0 tx=0x7f4498032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.697+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449803f030 con 0x7f44a81016e0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.698+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a819f640 con 0x7f44a81016e0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.698+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4498003ce0 con 0x7f44a81016e0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.698+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498035070 con 0x7f44a81016e0 2024-09-19T05:33:00.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.698+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a819fba0 con 0x7f44a81016e0 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.698+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4498042030 con 0x7f44a81016e0 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.699+0000 7f449e7fc640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 0x7f449403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.699+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a81905b0 con 0x7f44a81016e0 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.699+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f449806c940 con 0x7f44a81016e0 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.699+0000 7f44ac9ef640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 0x7f449403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:00.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.699+0000 7f44ac9ef640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 0x7f449403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f44a81984d0 tx=0x7f44a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:00.705 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.704+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f449803be20 con 0x7f44a81016e0 2024-09-19T05:33:00.888 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.886+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi047=y;smithi179=z", "target": ["mon-mgr", ""]}) -- 0x7f44a80626c0 con 0x7f4494039730 2024-09-19T05:33:00.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.891+0000 7f449e7fc640 1 -- 172.21.15.179:0/3723106524 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f44a80626c0 con 0x7f4494039730 2024-09-19T05:33:00.893 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-09-19T05:33:00.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 msgr2=0x7f449403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:00.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 0x7f449403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f44a81984d0 tx=0x7f44a4008040 comp rx=0 tx=0).stop 2024-09-19T05:33:00.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 msgr2=0x7f44a8196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:00.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4498005ab0 tx=0x7f4498032d00 comp rx=0 tx=0).stop 2024-09-19T05:33:00.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 shutdown_connections 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4494039730 0x7f449403bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44a8197970 0x7f44a819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a81974f0 0x7f44a819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 --2- 172.21.15.179:0/3723106524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44a81016e0 0x7f44a8196fb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 >> 172.21.15.179:0/3723106524 conn(0x7f44a80fce90 msgr2=0x7f44a80fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 shutdown_connections 2024-09-19T05:33:00.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:00.893+0000 7f44af454640 1 -- 172.21.15.179:0/3723106524 wait complete. 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: Reconfiguring daemon mon.a on smithi038 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:01.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:01.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: Reconfiguring daemon mon.a on smithi038 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:01.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: Reconfiguring mon.a (monmap changed)... 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: Reconfiguring daemon mon.a on smithi038 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:01.215 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:01.489 DEBUG:teuthology.orchestra.run.smithi047:mgr.y> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T05:33:01.491 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T05:33:01.494 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T05:33:01.494 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:33:01.494 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:33:01.515 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:33:01.515 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 05:32:17.863629060 +0000 2024-09-19T05:33:01.575 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 05:27:59.860851542 +0000 2024-09-19T05:33:01.576 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 05:27:59.860851542 +0000 2024-09-19T05:33:01.576 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 05:27:59.860851542 +0000 2024-09-19T05:33:01.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:33:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T05:33:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T05:33:01.645 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000322009 s, 1.6 MB/s 2024-09-19T05:33:01.646 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:33:01.706 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 05:32:17.896628465 +0000 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 05:28:00.198845467 +0000 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 05:28:00.198845467 +0000 2024-09-19T05:33:01.763 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 05:28:00.198845467 +0000 2024-09-19T05:33:01.764 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:33:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T05:33:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T05:33:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000295103 s, 1.7 MB/s 2024-09-19T05:33:01.831 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:33:01.889 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 05:32:17.930627851 +0000 2024-09-19T05:33:01.947 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 05:28:00.542839285 +0000 2024-09-19T05:33:01.948 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 05:28:00.542839285 +0000 2024-09-19T05:33:01.948 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 05:28:00.542839285 +0000 2024-09-19T05:33:01.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:33:02.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T05:33:02.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T05:33:02.013 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000378844 s, 1.4 MB/s 2024-09-19T05:33:02.014 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:33:02.072 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-09-19T05:33:02.127 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 05:32:17.965627219 +0000 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 05:28:00.873833337 +0000 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 05:28:00.873833337 +0000 2024-09-19T05:33:02.128 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 05:28:00.873833337 +0000 2024-09-19T05:33:02.128 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:33:02.177 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:01 smithi047 ceph-mon[35327]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:33:02.177 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:01 smithi047 ceph-mon[35327]: Reconfiguring daemon mon.b on smithi047 2024-09-19T05:33:02.177 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:01 smithi047 ceph-mon[35327]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi047=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:02.177 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:01 smithi047 ceph-mon[35327]: Saving service mgr spec with placement smithi038=x;smithi047=y;smithi179=z;count:3 2024-09-19T05:33:02.177 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:01 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:02.189 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:01 smithi038 ceph-mon[31227]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:33:02.189 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:01 smithi038 ceph-mon[31227]: Reconfiguring daemon mon.b on smithi047 2024-09-19T05:33:02.189 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:01 smithi038 ceph-mon[31227]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi047=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:02.189 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:01 smithi038 ceph-mon[31227]: Saving service mgr spec with placement smithi038=x;smithi047=y;smithi179=z;count:3 2024-09-19T05:33:02.190 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:01 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:02.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-19T05:33:02.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-19T05:33:02.192 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000277872 s, 1.8 MB/s 2024-09-19T05:33:02.193 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:33:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:01 smithi179 ceph-mon[35781]: Reconfiguring mon.b (monmap changed)... 2024-09-19T05:33:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:01 smithi179 ceph-mon[35781]: Reconfiguring daemon mon.b on smithi047 2024-09-19T05:33:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:01 smithi179 ceph-mon[35781]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi047=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:01 smithi179 ceph-mon[35781]: Saving service mgr spec with placement smithi038=x;smithi047=y;smithi179=z;count:3 2024-09-19T05:33:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:01 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:02.250 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-19T05:33:02.251 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:33:02.267 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:33:02.267 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-19T05:33:02.324 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-19 05:32:51.680069513 +0000 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-19 05:28:19.936127464 +0000 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-19 05:28:19.936127464 +0000 2024-09-19T05:33:02.325 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-19 05:28:19.936127464 +0000 2024-09-19T05:33:02.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:33:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-19T05:33:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-19T05:33:02.388 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000110219 s, 4.6 MB/s 2024-09-19T05:33:02.389 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:33:02.446 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-19 05:32:51.680069513 +0000 2024-09-19T05:33:02.502 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-19 05:28:20.260120421 +0000 2024-09-19T05:33:02.503 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-19 05:28:20.260120421 +0000 2024-09-19T05:33:02.503 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-19 05:28:20.260120421 +0000 2024-09-19T05:33:02.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:33:02.565 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-19T05:33:02.565 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-19T05:33:02.566 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.1126e-05 s, 5.6 MB/s 2024-09-19T05:33:02.567 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:33:02.622 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-19 05:32:51.680069513 +0000 2024-09-19T05:33:02.678 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-19 05:28:20.575113573 +0000 2024-09-19T05:33:02.679 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-19 05:28:20.575113573 +0000 2024-09-19T05:33:02.679 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-19 05:28:20.575113573 +0000 2024-09-19T05:33:02.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:33:02.741 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-19T05:33:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-19T05:33:02.742 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.3866e-05 s, 5.5 MB/s 2024-09-19T05:33:02.743 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:33:02.799 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-19 05:32:51.681069491 +0000 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-19 05:28:20.891106704 +0000 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-19 05:28:20.891106704 +0000 2024-09-19T05:33:02.855 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-19 05:28:20.891106704 +0000 2024-09-19T05:33:02.856 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:33:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-19T05:33:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-19T05:33:02.918 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000103718 s, 4.9 MB/s 2024-09-19T05:33:02.919 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:33:02.976 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-09-19T05:33:02.976 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:33:02.992 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:33:02.992 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-09-19 05:32:42.909951124 +0000 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-09-19 05:28:17.260220290 +0000 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-09-19 05:28:17.260220290 +0000 2024-09-19T05:33:03.048 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-09-19 05:28:17.260220290 +0000 2024-09-19T05:33:03.049 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:33:03.112 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-09-19T05:33:03.112 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-09-19T05:33:03.112 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000265955 s, 1.9 MB/s 2024-09-19T05:33:03.113 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:33:03.170 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-09-19 05:32:42.910951103 +0000 2024-09-19T05:33:03.225 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-09-19 05:28:17.604212960 +0000 2024-09-19T05:33:03.226 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-09-19 05:28:17.604212960 +0000 2024-09-19T05:33:03.226 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-09-19 05:28:17.604212960 +0000 2024-09-19T05:33:03.226 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:33:03.288 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-09-19T05:33:03.289 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-09-19T05:33:03.289 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.00019503 s, 2.6 MB/s 2024-09-19T05:33:03.290 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:33:03.347 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-09-19T05:33:03.401 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-09-19 05:32:42.910951103 +0000 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-09-19 05:28:17.931206006 +0000 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-09-19 05:28:17.931206006 +0000 2024-09-19T05:33:03.402 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-09-19 05:28:17.931206006 +0000 2024-09-19T05:33:03.402 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:33:03.465 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-09-19T05:33:03.465 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-09-19T05:33:03.465 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000251542 s, 2.0 MB/s 2024-09-19T05:33:03.466 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:33:03.522 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-09-19T05:33:03.567 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:03.567 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.567 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.567 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:33:03.567 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:03.568 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:03.568 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:03.568 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:03 smithi179 ceph-mon[35781]: Reconfiguring daemon mon.c on smithi179 2024-09-19T05:33:03.578 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-09-19 05:32:42.911951082 +0000 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-09-19 05:28:18.262198967 +0000 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-09-19 05:28:18.262198967 +0000 2024-09-19T05:33:03.580 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-09-19 05:28:18.262198967 +0000 2024-09-19T05:33:03.580 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:33:03.642 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-09-19T05:33:03.642 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-09-19T05:33:03.643 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000196404 s, 2.6 MB/s 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:03 smithi038 ceph-mon[31227]: Reconfiguring daemon mon.c on smithi179 2024-09-19T05:33:03.645 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:33:03.701 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-09-19T05:33:03.702 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: Reconfiguring mon.c (monmap changed)... 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T05:33:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T05:33:03.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:03.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:03 smithi047 ceph-mon[35327]: Reconfiguring daemon mon.c on smithi179 2024-09-19T05:33:03.960 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:33:04.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:04.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:04.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:04 smithi038 ceph-mon[31227]: Deploying daemon mgr.z on smithi179 2024-09-19T05:33:04.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:04.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:04.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:04 smithi179 ceph-mon[35781]: Deploying daemon mgr.z on smithi179 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:04.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:04.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:04.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:04.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:04.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:04.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:04 smithi047 ceph-mon[35327]: Deploying daemon mgr.z on smithi179 2024-09-19T05:33:05.447 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 systemd[1]: Starting Ceph mgr.z for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:33:05.845 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 podman[37569]: 2024-09-19 05:33:05.558444382 +0000 UTC m=+0.100760953 container create 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:33:05.845 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 podman[37569]: 2024-09-19 05:33:05.473407288 +0000 UTC m=+0.015723868 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:33:05.846 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 podman[37569]: 2024-09-19 05:33:05.768982533 +0000 UTC m=+0.311299105 container init 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:33:05.846 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 podman[37569]: 2024-09-19 05:33:05.772125486 +0000 UTC m=+0.314442059 container start 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:33:05.846 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 ceph-mgr[37588]: -- 172.21.15.179:0/3651540102 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558921d070e0 con 0x558921ce4800 2024-09-19T05:33:05.846 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 bash[37569]: 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab 2024-09-19T05:33:06.101 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 systemd[1]: Started Ceph mgr.z for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:33:06.101 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:05.894+0000 7f791bb85040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:33:06.101 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:05.954+0000 7f791bb85040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:33:06.101 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.099+0000 7f791bb85040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:33:06.215 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:33:06.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-09-19T05:33:06.447 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.164+0000 7f791bb85040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:33:06.448 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.309+0000 7f791bb85040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:33:06.492 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:33:06.761 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.445+0000 7f791bb85040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:33:06.761 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.759+0000 7f791bb85040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:33:06.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 -- 172.21.15.38:0/3872787588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280076220 msgr2=0x7fe280074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:06.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 --2- 172.21.15.38:0/3872787588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280076220 0x7fe280074680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe270009900 tx=0x7fe27002a410 comp rx=0 tx=0).stop 2024-09-19T05:33:06.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 -- 172.21.15.38:0/3872787588 shutdown_connections 2024-09-19T05:33:06.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 --2- 172.21.15.38:0/3872787588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280109a80 0x7fe28010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 --2- 172.21.15.38:0/3872787588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280074c50 0x7fe2800750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 --2- 172.21.15.38:0/3872787588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280076220 0x7fe280074680 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 -- 172.21.15.38:0/3872787588 >> 172.21.15.38:0/3872787588 conn(0x7fe2800fd390 msgr2=0x7fe2800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 -- 172.21.15.38:0/3872787588 shutdown_connections 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.951+0000 7fe285800640 1 -- 172.21.15.38:0/3872787588 wait complete. 2024-09-19T05:33:06.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 Processor -- start 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 -- start start 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280076220 0x7fe28019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280109a80 0x7fe2801a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010fad0 con 0x7fe280074c50 2024-09-19T05:33:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010f950 con 0x7fe280109a80 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.952+0000 7fe285800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010fc50 con 0x7fe280076220 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46212/0 (socket says 172.21.15.38:46212) 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 -- 172.21.15.38:0/3533020971 learned_addr learned my addr 172.21.15.38:0/3533020971 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe277fff640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280076220 0x7fe28019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27f7fe640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280109a80 0x7fe2801a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 -- 172.21.15.38:0/3533020971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280076220 msgr2=0x7fe28019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280076220 0x7fe28019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 -- 172.21.15.38:0/3533020971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280109a80 msgr2=0x7fe2801a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280109a80 0x7fe2801a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 -- 172.21.15.38:0/3533020971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2801a1d30 con 0x7fe280074c50 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27effd640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe270031440 tx=0x7fe270032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27003f030 con 0x7fe280074c50 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe270003ce0 con 0x7fe280074c50 2024-09-19T05:33:06.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2801a1fc0 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270035070 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.953+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2801a23c0 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.954+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe270042030 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.954+0000 7fe27cff9640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 0x7fe26c03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.955+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe27006cca0 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.955+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe280108800 con 0x7fe280074c50 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.955+0000 7fe277fff640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 0x7fe26c03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:06.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.955+0000 7fe277fff640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 0x7fe26c03b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe28019b620 tx=0x7fe26800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:06.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:06.960+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe27003bdc0 con 0x7fe280074c50 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:07.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:06 smithi038 ceph-mon[31227]: Deploying daemon mgr.y on smithi047 2024-09-19T05:33:07.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:07.144+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe280062630 con 0x7fe26c039500 2024-09-19T05:33:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-mon[35781]: Deploying daemon mgr.y on smithi047 2024-09-19T05:33:07.199 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:06.819+0000 7f791bb85040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:33:07.199 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:07.003+0000 7f791bb85040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:06 smithi047 ceph-mon[35327]: Deploying daemon mgr.y on smithi047 2024-09-19T05:33:07.925 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:07 smithi047 systemd[1]: Starting Ceph mgr.y for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:33:08.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:07 smithi038 ceph-mon[31227]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:08.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:08.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:08.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-mon[35781]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:08.198 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:07.880+0000 7f791bb85040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:33:08.198 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:07.948+0000 7f791bb85040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:33:08.219 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:07 smithi047 ceph-mon[35327]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:08.219 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:08.219 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:08.219 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:08.220 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 podman[36679]: 2024-09-19 05:33:08.032942469 +0000 UTC m=+0.106557467 container create e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T05:33:08.220 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 podman[36679]: 2024-09-19 05:33:07.942641189 +0000 UTC m=+0.016256145 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:33:08.220 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 podman[36679]: 2024-09-19 05:33:08.218461305 +0000 UTC m=+0.292076262 container init e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 podman[36679]: 2024-09-19 05:33:08.221256773 +0000 UTC m=+0.294871730 container start e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-mgr[36699]: -- 172.21.15.47:0/2470924517 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ecce1190e0 con 0x55ecce0f7000 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 bash[36679]: e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 systemd[1]: Started Ceph mgr.y for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.343+0000 7f4ddb4ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:33:08.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.405+0000 7f4ddb4ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:33:08.947 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:08.675+0000 7f791bb85040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:33:08.947 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:08.801+0000 7f791bb85040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:33:08.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.562+0000 7f4ddb4ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:33:08.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.633+0000 7f4ddb4ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:33:08.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.793+0000 7f4ddb4ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:33:08.954 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:08.952+0000 7f4ddb4ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:33:09.256 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:08.959+0000 7f791bb85040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:33:09.256 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.020+0000 7f791bb85040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:33:09.256 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.184+0000 7f791bb85040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:33:09.256 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.253+0000 7f791bb85040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:09.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:09.286+0000 7f4ddb4ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:33:09.477 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:09.349+0000 7f4ddb4ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.552 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:09.553 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.315+0000 7f791bb85040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:33:09.553 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.478+0000 7f791bb85040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:33:09.553 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.550+0000 7f791bb85040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:33:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:09.823 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.617+0000 7f791bb85040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:33:09.823 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.703+0000 7f791bb85040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:33:09.823 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.761+0000 7f791bb85040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:33:09.823 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:09.821+0000 7f791bb85040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:33:09.975 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:09.554+0000 7f4ddb4ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]: dispatch 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]': finished 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1559415556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:10.697 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z[37584]: 2024-09-19T05:33:10.604+0000 7f791bb85040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:33:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]: dispatch 2024-09-19T05:33:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]': finished 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.699 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1559415556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:10.474+0000 7f4ddb4ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:10.545+0000 7f4ddb4ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]: dispatch 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1668897138' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d08454b-46b5-4ad6-8e7f-3c79c155e73d"}]': finished 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:10.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1559415556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:11.409+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7fe2700438e0 con 0x7fe280074c50 2024-09-19T05:33:11.568 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.295+0000 7f4ddb4ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:33:11.568 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.416+0000 7f4ddb4ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:33:11.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:11.568 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: Reconfiguring daemon mgr.x on smithi038 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: Standby manager daemon z started 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:33:11.569 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:33:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: Reconfiguring daemon mgr.x on smithi038 2024-09-19T05:33:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: Standby manager daemon z started 2024-09-19T05:33:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:33:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:33:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:11 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:11.855 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.564+0000 7f4ddb4ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:33:11.855 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.628+0000 7f4ddb4ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:33:11.855 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.783+0000 7f4ddb4ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:33:11.855 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.853+0000 7f4ddb4ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: Reconfiguring daemon mgr.x on smithi038 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: Standby manager daemon z started 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:33:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:11 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.179:0/2127205487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:12.149 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:11.915+0000 7f4ddb4ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:33:12.149 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.079+0000 7f4ddb4ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:33:12.149 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.147+0000 7f4ddb4ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:33:12.413 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.216+0000 7f4ddb4ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:33:12.413 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.302+0000 7f4ddb4ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:33:12.414 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.364+0000 7f4ddb4ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:33:12.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:12 smithi179 ceph-mon[35781]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T05:33:12.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:33:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-mon[35327]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T05:33:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:33:12.726 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:12.426+0000 7f4ddb4ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:33:12.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:12 smithi038 ceph-mon[31227]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T05:33:12.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:33:13.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:13.421+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fe27003aca0 con 0x7fe280074c50 2024-09-19T05:33:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: Standby manager daemon y started 2024-09-19T05:33:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:33:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:33:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:13 smithi179 ceph-mon[35781]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y[36695]: 2024-09-19T05:33:13.249+0000 7f4ddb4ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: Standby manager daemon y started 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:33:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:13 smithi047 ceph-mon[35327]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: Standby manager daemon y started 2024-09-19T05:33:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:33:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:33:13.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:33:13.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:13 smithi038 ceph-mon[31227]: from='mgr.? 172.21.15.47:0/2644279592' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:33:14.633 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:14 smithi038 ceph-mon[31227]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T05:33:14.633 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:33:14.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:14 smithi179 ceph-mon[35781]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T05:33:14.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:33:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:14 smithi047 ceph-mon[35327]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T05:33:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:33:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:15 smithi038 ceph-mon[31227]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:15 smithi179 ceph-mon[35781]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:15 smithi047 ceph-mon[35327]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:17.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:17.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:17.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:17.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:17.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:17.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:17.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:19 smithi038 ceph-mon[31227]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:33:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:19 smithi038 ceph-mon[31227]: Deploying daemon osd.0 on smithi038 2024-09-19T05:33:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:19 smithi179 ceph-mon[35781]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:33:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:19 smithi179 ceph-mon[35781]: Deploying daemon osd.0 on smithi038 2024-09-19T05:33:19.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:19 smithi047 ceph-mon[35327]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:19.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T05:33:19.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:19.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:19 smithi047 ceph-mon[35327]: Deploying daemon osd.0 on smithi038 2024-09-19T05:33:21.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:21 smithi038 ceph-mon[31227]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:21 smithi179 ceph-mon[35781]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:21.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:21 smithi047 ceph-mon[35327]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:23.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:23 smithi038 ceph-mon[31227]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:23.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:23 smithi179 ceph-mon[35781]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:23.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:23 smithi047 ceph-mon[35327]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:25.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:25 smithi038 ceph-mon[31227]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:25.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:25 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:25.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:25 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:25.335 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:25 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:25 smithi179 ceph-mon[35781]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:25 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:25 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:25 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:25 smithi047 ceph-mon[35327]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:25 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:25 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:25 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:27.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:27 smithi038 ceph-mon[31227]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:27 smithi179 ceph-mon[35781]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:27 smithi047 ceph-mon[35327]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:28.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.014+0000 7fe27cff9640 1 -- 172.21.15.38:0/3533020971 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe280062630 con 0x7fe26c039500 2024-09-19T05:33:28.017 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-09-19T05:33:28.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 msgr2=0x7fe26c03b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:28.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 0x7fe26c03b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe28019b620 tx=0x7fe26800b390 comp rx=0 tx=0).stop 2024-09-19T05:33:28.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 msgr2=0x7fe28019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:28.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe270031440 tx=0x7fe270032e20 comp rx=0 tx=0).stop 2024-09-19T05:33:28.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 shutdown_connections 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe26c039500 0x7fe26c03b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe280109a80 0x7fe2801a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe280076220 0x7fe28019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 --2- 172.21.15.38:0/3533020971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe280074c50 0x7fe28019a100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.017+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 >> 172.21.15.38:0/3533020971 conn(0x7fe2800fd390 msgr2=0x7fe2801059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.018+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 shutdown_connections 2024-09-19T05:33:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:33:28.018+0000 7fe285800640 1 -- 172.21.15.38:0/3533020971 wait complete. 2024-09-19T05:33:28.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:28.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:28.682 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T05:33:28.684 INFO:tasks.cephadm:Deploying osd.1 on smithi047 with /dev/vg_nvme/lv_4... 2024-09-19T05:33:28.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:33:28.928 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:33:29.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:29 smithi047 ceph-mon[35327]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:29.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:29 smithi038 ceph-mon[31227]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:29.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:29 smithi179 ceph-mon[35781]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:31.085 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:33:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:33:30.657+0000 7f8653ee6640 -1 osd.0 0 log_to_monitors true 2024-09-19T05:33:31.272 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:33:31.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-09-19T05:33:31.314 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:31 smithi047 ceph-mon[35327]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:31.315 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:31 smithi047 ceph-mon[35327]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:33:31.386 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:31 smithi038 ceph-mon[31227]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:31.386 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:31 smithi038 ceph-mon[31227]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:33:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:31 smithi179 ceph-mon[35781]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:31 smithi179 ceph-mon[35781]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T05:33:31.558 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:33:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 -- 172.21.15.47:0/3492757912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff088103870 msgr2=0x7ff08810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 --2- 172.21.15.47:0/3492757912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff088103870 0x7ff08810a100 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff080009ca0 tx=0x7ff08002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 -- 172.21.15.47:0/3492757912 shutdown_connections 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 --2- 172.21.15.47:0/3492757912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff088103870 0x7ff08810a100 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 --2- 172.21.15.47:0/3492757912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 --2- 172.21.15.47:0/3492757912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880ffed0 0x7ff0881002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 -- 172.21.15.47:0/3492757912 >> 172.21.15.47:0/3492757912 conn(0x7ff0880fba40 msgr2=0x7ff0880fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 -- 172.21.15.47:0/3492757912 shutdown_connections 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.954+0000 7ff08d408640 1 -- 172.21.15.47:0/3492757912 wait complete. 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 Processor -- start 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 -- start start 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff088103870 0x7ff0881013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810d890 con 0x7ff088102eb0 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810d710 con 0x7ff0880ffed0 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff08d408640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810da10 con 0x7ff088103870 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.955+0000 7ff086ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48136/0 (socket says 172.21.15.47:48136) 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 -- 172.21.15.47:0/909653089 learned_addr learned my addr 172.21.15.47:0/909653089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff07e7fc640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff0877fe640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff088103870 0x7ff0881013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 -- 172.21.15.47:0/909653089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff088103870 msgr2=0x7ff0881013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff088103870 0x7ff0881013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 -- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 msgr2=0x7ff088100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:31.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 -- 172.21.15.47:0/909653089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088101c60 con 0x7ff0880ffed0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff07e7fc640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff086ffd640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c970 tx=0x7ff07800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078007c10 con 0x7ff0880ffed0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0881a1ff0 con 0x7ff0880ffed0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff078007db0 con 0x7ff0880ffed0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800f8a0 con 0x7ff0880ffed0 2024-09-19T05:33:31.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.956+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881a2440 con 0x7ff0880ffed0 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.957+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff088109060 con 0x7ff0880ffed0 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.958+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff07800fa40 con 0x7ff0880ffed0 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.959+0000 7ff084ff9640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 0x7ff0740a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.959+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7ff0780ce1d0 con 0x7ff0880ffed0 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.959+0000 7ff07e7fc640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 0x7ff0740a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:31.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.960+0000 7ff07e7fc640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 0x7ff0740a56f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff088102590 tx=0x7ff07000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:31.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:31.963+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff07809cc10 con 0x7ff0880ffed0 2024-09-19T05:33:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:32.152+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff088062630 con 0x7ff0740a3230 2024-09-19T05:33:32.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:33:32.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi038 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:33:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:32 smithi179 ceph-mon[35781]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi038 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T05:33:32.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:32.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:32.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:33:32.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:32 smithi047 ceph-mon[35327]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:33:32.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:33:32.167+0000 7f864fe69640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T05:33:32.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:33:32.174+0000 7f864bc91640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:33:32.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T05:33:32.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T05:33:32.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi038 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: from='osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:33:32.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:32 smithi038 ceph-mon[31227]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T05:33:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:33.452 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.453 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:34.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:33:34.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:33:34.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:34.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] boot 2024-09-19T05:33:34.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:33:34.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] boot 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:33:34.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: osd.0 [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] boot 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T05:33:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/3098874568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]': finished 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:35 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/2090537711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/3098874568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]': finished 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:35 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/2090537711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/3098874568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]: dispatch 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7156f9e6-19c1-4422-89f5-97209c281838"}]': finished 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T05:33:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:35.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:35 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/2090537711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:33:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:37 smithi179 ceph-mon[35781]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:37.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:37 smithi047 ceph-mon[35327]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:37 smithi038 ceph-mon[31227]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:39.435 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:39 smithi047 ceph-mon[35327]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:39 smithi038 ceph-mon[31227]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:39 smithi179 ceph-mon[35781]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:41.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:41 smithi047 ceph-mon[35327]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:41.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:41 smithi038 ceph-mon[31227]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:41 smithi179 ceph-mon[35781]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:42.389 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:33:42.390 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:42.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:33:42.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:42.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T05:33:42.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:43.376 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:43 smithi047 ceph-mon[35327]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:43.376 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:43 smithi047 ceph-mon[35327]: Deploying daemon osd.1 on smithi047 2024-09-19T05:33:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:43 smithi038 ceph-mon[31227]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:43 smithi038 ceph-mon[31227]: Deploying daemon osd.1 on smithi047 2024-09-19T05:33:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:43 smithi179 ceph-mon[35781]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:43 smithi179 ceph-mon[35781]: Deploying daemon osd.1 on smithi047 2024-09-19T05:33:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:45 smithi047 ceph-mon[35327]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:45.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:45 smithi038 ceph-mon[31227]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:45 smithi179 ceph-mon[35781]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:47.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:47 smithi047 ceph-mon[35327]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:47 smithi038 ceph-mon[31227]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:47.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:47 smithi179 ceph-mon[35781]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:49.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:49 smithi047 ceph-mon[35327]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:49.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:49.362 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:49.362 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:49.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:49 smithi179 ceph-mon[35781]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:49.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:49.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:49.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:49 smithi038 ceph-mon[31227]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.480+0000 7ff084ff9640 1 -- 172.21.15.47:0/909653089 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff088062630 con 0x7ff0740a3230 2024-09-19T05:33:52.483 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 1 on host 'smithi047' 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 msgr2=0x7ff0740a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 0x7ff0740a56f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff088102590 tx=0x7ff07000b410 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 msgr2=0x7ff088102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff07800c970 tx=0x7ff07800ce30 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 shutdown_connections 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff0740a3230 0x7ff0740a56f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff088103870 0x7ff0881013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff088102eb0 0x7ff088100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 --2- 172.21.15.47:0/909653089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0880ffed0 0x7ff088102810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 >> 172.21.15.47:0/909653089 conn(0x7ff0880fba40 msgr2=0x7ff088106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 shutdown_connections 2024-09-19T05:33:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:33:52.483+0000 7ff08d408640 1 -- 172.21.15.47:0/909653089 wait complete. 2024-09-19T05:33:53.094 DEBUG:teuthology.orchestra.run.smithi047:osd.1> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T05:33:53.136 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-09-19T05:33:53.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T05:33:53.359 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:53 smithi047 ceph-mon[35327]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:53.359 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:53.360 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:53.360 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:53.400 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:33:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:53 smithi179 ceph-mon[35781]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:53.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:53 smithi038 ceph-mon[31227]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:33:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:54.476 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:33:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:33:54.302+0000 7fd004496640 -1 osd.1 0 log_to_monitors true 2024-09-19T05:33:55.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:55 smithi179 ceph-mon[35781]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:55.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:55 smithi179 ceph-mon[35781]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:55 smithi179 ceph-mon[35781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:55 smithi047 ceph-mon[35327]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:55.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:55 smithi047 ceph-mon[35327]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:55 smithi047 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:55 smithi038 ceph-mon[31227]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:55 smithi038 ceph-mon[31227]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:55 smithi038 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T05:33:55.817 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:33:55.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-09-19T05:33:56.121 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.503 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:33:56.504 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.566+0000 7f8877150640 1 -- 172.21.15.179:0/1459399019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870100b00 msgr2=0x7f8870100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:56.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.566+0000 7f8877150640 1 --2- 172.21.15.179:0/1459399019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870100b00 0x7f8870100f00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8868009900 tx=0x7f886802a410 comp rx=0 tx=0).stop 2024-09-19T05:33:56.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- 172.21.15.179:0/1459399019 shutdown_connections 2024-09-19T05:33:56.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- 172.21.15.179:0/1459399019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f8870108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- 172.21.15.179:0/1459399019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870101d00 0x7f8870102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- 172.21.15.179:0/1459399019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870100b00 0x7f8870100f00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- 172.21.15.179:0/1459399019 >> 172.21.15.179:0/1459399019 conn(0x7f88700fc290 msgr2=0x7f88700fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- 172.21.15.179:0/1459399019 shutdown_connections 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- 172.21.15.179:0/1459399019 wait complete. 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 Processor -- start 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- start start 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870100b00 0x7f887006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f887019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f887010c760 con 0x7f8870100b00 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f887010c5e0 con 0x7f88701026c0 2024-09-19T05:33:56.594 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f8877150640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f887010c8e0 con 0x7f8870101d00 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f886ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f886ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47330/0 (socket says 172.21.15.179:47330) 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.567+0000 7f886ffff640 1 -- 172.21.15.179:0/1494207222 learned_addr learned my addr 172.21.15.179:0/1494207222 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f8874eec640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870100b00 0x7f887006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f88756ed640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f887019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 -- 172.21.15.179:0/1494207222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 msgr2=0x7f887019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f887019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 -- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870100b00 msgr2=0x7f887006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870100b00 0x7f887006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 -- 172.21.15.179:0/1494207222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887019fba0 con 0x7f8870101d00 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f88756ed640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f887019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:33:56.595 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886ffff640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f885800eab0 tx=0x7f885800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885800ce40 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887019fe30 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8858011950 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858010ae0 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.568+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701a0330 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.569+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8870107fa0 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.570+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f8858011b00 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.571+0000 7f886dffb640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 0x7f885c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.571+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f88580ce710 con 0x7f8870101d00 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.571+0000 7f8874eec640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 0x7f885c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.572+0000 7f8874eec640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 0x7f885c0a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8868006010 tx=0x7f8868000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:33:56.596 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.575+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8858016030 con 0x7f8870101d00 2024-09-19T05:33:56.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:33:56.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:33:56.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:33:56.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.726 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:33:56.250+0000 7fd000419640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T05:33:56.726 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:33:56.254+0000 7fcffc241640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:56.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:33:56.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:33:56.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:33:56.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T05:33:56.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:56.756 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:33:56.753+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8870062630 con 0x7f885c0a31b0 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi047 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi047 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi047 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:33:57.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:58.313 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:33:58.313 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:33:58.313 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:58.314 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] boot 2024-09-19T05:33:58.314 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:33:58.314 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:58 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] boot 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:33:58.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:58 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: osd.1 [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] boot 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T05:33:58.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:58 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:33:59.546 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:59.546 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T05:33:59.546 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/457060216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.547 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.547 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]': finished 2024-09-19T05:33:59.547 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:33:59.547 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:33:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/457060216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]': finished 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:33:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:33:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/457060216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]: dispatch 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94fbc241-9df4-4189-9369-72b359a64f08"}]': finished 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T05:33:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:33:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:00.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:00 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/3352939282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:00.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:00 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/3352939282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:00.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:00 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/3352939282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:01.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:01 smithi038 ceph-mon[31227]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:01 smithi179 ceph-mon[35781]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:01.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:01 smithi047 ceph-mon[35327]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:03 smithi038 ceph-mon[31227]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:03 smithi179 ceph-mon[35781]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:03.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:03 smithi047 ceph-mon[35327]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:05 smithi179 ceph-mon[35781]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:05 smithi038 ceph-mon[31227]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:05 smithi047 ceph-mon[35327]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:07 smithi038 ceph-mon[31227]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:34:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:07.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:07 smithi179 ceph-mon[35781]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:07.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:34:07.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:07 smithi047 ceph-mon[35327]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T05:34:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:08.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:08 smithi038 ceph-mon[31227]: Deploying daemon osd.2 on smithi179 2024-09-19T05:34:08.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:08 smithi179 ceph-mon[35781]: Deploying daemon osd.2 on smithi179 2024-09-19T05:34:08.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:08 smithi047 ceph-mon[35327]: Deploying daemon osd.2 on smithi179 2024-09-19T05:34:09.600 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:09 smithi179 ceph-mon[35781]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:09 smithi038 ceph-mon[31227]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:09.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:09 smithi047 ceph-mon[35327]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:11 smithi038 ceph-mon[31227]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:11 smithi179 ceph-mon[35781]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:11.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:11 smithi047 ceph-mon[35327]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:13 smithi038 ceph-mon[31227]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:13 smithi179 ceph-mon[35781]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:13 smithi047 ceph-mon[35327]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:14.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:14.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:14.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:14.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:14.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:14.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:15.561 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:15 smithi179 ceph-mon[35781]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:15 smithi038 ceph-mon[31227]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:15 smithi047 ceph-mon[35327]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:16.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:16.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.254+0000 7f886dffb640 1 -- 172.21.15.179:0/1494207222 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8870062630 con 0x7f885c0a31b0 2024-09-19T05:34:17.258 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 msgr2=0x7f885c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 0x7f885c0a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8868006010 tx=0x7f8868000bc0 comp rx=0 tx=0).stop 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 msgr2=0x7f887019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f885800eab0 tx=0x7f885800ef70 comp rx=0 tx=0).stop 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 shutdown_connections 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f885c0a31b0 0x7f885c0a5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88701026c0 0x7f887019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:17.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8870101d00 0x7f887019cc70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:17.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 --2- 172.21.15.179:0/1494207222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8870100b00 0x7f887006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:17.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 >> 172.21.15.179:0/1494207222 conn(0x7f88700fc290 msgr2=0x7f8870105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:34:17.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 shutdown_connections 2024-09-19T05:34:17.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:34:17.257+0000 7f8877150640 1 -- 172.21.15.179:0/1494207222 wait complete. 2024-09-19T05:34:17.844 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:17.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.845 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.871 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T05:34:17.873 INFO:tasks.cephadm:Deploying osd.3 on smithi038 with /dev/vg_nvme/lv_3... 2024-09-19T05:34:17.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:34:17.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:17.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:17.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:34:18.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:18.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:18.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:19.515 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:19 smithi038 ceph-mon[31227]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:19 smithi179 ceph-mon[35781]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:19.698 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:34:19.445+0000 7efc53580640 -1 osd.2 0 log_to_monitors true 2024-09-19T05:34:19.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:19 smithi047 ceph-mon[35327]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:20.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:20 smithi038 ceph-mon[31227]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:20 smithi038 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:20 smithi179 ceph-mon[35781]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:20 smithi179 ceph-mon[35781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:20 smithi047 ceph-mon[35327]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:20 smithi047 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T05:34:20.729 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:34:20.757 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-09-19T05:34:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:21.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:21.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 -- 172.21.15.38:0/797860086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 msgr2=0x7f99ac107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 --2- 172.21.15.38:0/797860086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac107420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009900 tx=0x7f99a002a410 comp rx=0 tx=0).stop 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 -- 172.21.15.38:0/797860086 shutdown_connections 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 --2- 172.21.15.38:0/797860086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac107960 0x7f99ac109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 --2- 172.21.15.38:0/797860086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac107420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 --2- 172.21.15.38:0/797860086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac102700 0x7f99ac104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:21.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 -- 172.21.15.38:0/797860086 >> 172.21.15.38:0/797860086 conn(0x7f99ac0fc280 msgr2=0x7f99ac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 -- 172.21.15.38:0/797860086 shutdown_connections 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.482+0000 7f99b24d0640 1 -- 172.21.15.38:0/797860086 wait complete. 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 Processor -- start 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 -- start start 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 0x7f99ac102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 0x7f99ac100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d850 con 0x7f99ac105030 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d6d0 con 0x7f99ac102700 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99b24d0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac10d9d0 con 0x7f99ac107960 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.483+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:21.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60472/0 (socket says 172.21.15.38:60472) 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 -- 172.21.15.38:0/1142425605 learned_addr learned my addr 172.21.15.38:0/1142425605 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99abfff640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 0x7f99ac102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99b0a6d640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 0x7f99ac100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 -- 172.21.15.38:0/1142425605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 msgr2=0x7f99ac100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 0x7f99ac100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 -- 172.21.15.38:0/1142425605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 msgr2=0x7f99ac102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 0x7f99ac102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 -- 172.21.15.38:0/1142425605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1014b0 con 0x7f99ac105030 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99abfff640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 0x7f99ac102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99b0a6d640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 0x7f99ac100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99ab7fe640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99a0031610 tx=0x7f99a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:34:21.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a003f030 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0003ad0 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac1a2060 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0035070 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.484+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a2510 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.486+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ac0fdd70 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.487+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f99a0042030 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.487+0000 7f99a97fa640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 0x7f99940a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.488+0000 7f99abfff640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 0x7f99940a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.488+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f99a00ee900 con 0x7f99ac105030 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.488+0000 7f99abfff640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 0x7f99940a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f999c005fd0 tx=0x7f999c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:34:21.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.492+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99a00bcd70 con 0x7f99ac105030 2024-09-19T05:34:21.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:21.678+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f99ac062630 con 0x7f99940a31b0 2024-09-19T05:34:21.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:34:21.273+0000 7efc4f503640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T05:34:21.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:34:21.277+0000 7efc4b32b640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:21.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:21.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:21.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:21.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:22.366 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi179 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi179 to 17088M 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:22.367 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:22.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi179 2024-09-19T05:34:22.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi179 to 17088M 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:22.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi179 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi179 to 17088M 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:22.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] boot 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:34:23.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] boot 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:34:23.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: osd.2 [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] boot 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T05:34:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:34:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:34:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]: dispatch 2024-09-19T05:34:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]': finished 2024-09-19T05:34:24.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:34:24.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:24.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:24.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]: dispatch 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]': finished 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:24.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:24.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T05:34:24.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:24.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]: dispatch 2024-09-19T05:34:24.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:24.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3764914694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c145d0a-468b-40cd-bc68-2214559e2fdf"}]': finished 2024-09-19T05:34:24.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T05:34:24.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:24.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:34:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:25 smithi038 ceph-mon[31227]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:25 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/93344843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:25 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:25 smithi038 ceph-mon[31227]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:34:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:25 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:25 smithi179 ceph-mon[35781]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:25 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/93344843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:25 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:25.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:25 smithi179 ceph-mon[35781]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:34:25.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:25 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:25 smithi047 ceph-mon[35327]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:25 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/93344843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:25 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:34:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:25 smithi047 ceph-mon[35327]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T05:34:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:25 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:26.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:25 smithi038 sudo[47465]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:26.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:25 smithi038 sudo[47465]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:26.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:25 smithi038 sudo[47465]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:26.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:25 smithi038 sudo[47465]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:26.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:26.314+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f99a0003c80 con 0x7f99ac105030 2024-09-19T05:34:26.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:26 smithi038 sudo[47834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:26.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:26 smithi038 sudo[47834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:26.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:26 smithi038 sudo[47834]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:26.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:34:26 smithi038 sudo[47834]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:26.747 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43487]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:26.747 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43487]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:26.748 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43487]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:26.748 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43487]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:27.205 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43490]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:27.205 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43490]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:27.206 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43490]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:27.206 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:34:26 smithi047 sudo[43490]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:34:27.277 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.278 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:27.278 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:27.278 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44769]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:27.278 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44769]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:34:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T05:34:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:27.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44772]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:27.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44772]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:27.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44772]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:27.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:34:27 smithi179 sudo[44772]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:28.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 sudo[48209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 sudo[48209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 sudo[48209]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:27 smithi038 sudo[48209]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:28.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:28 smithi038 sudo[48214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:28.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:28 smithi038 sudo[48214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:28.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:28 smithi038 sudo[48214]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:28.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:28 smithi038 sudo[48214]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:28.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44775]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:28.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44775]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:28.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44775]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:28.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44775]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44778]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44778]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44778]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:28 smithi179 sudo[44778]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:29.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:28 smithi047 sudo[43493]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T05:34:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:28 smithi047 sudo[43493]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:28 smithi047 sudo[43493]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:28 smithi047 sudo[43493]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 sudo[43496]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 sudo[43496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 sudo[43496]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 sudo[43496]: pam_unix(sudo:session): session closed for user root 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:29.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:29.730 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:29.730 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.731 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.731 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:29.732 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:29.732 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:29.733 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:30.497 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:30.497 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.497 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:30.498 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.498 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:30.498 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:30.498 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:30.498 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:30 smithi047 ceph-mon[35327]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:30.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:30.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:30.565 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:30 smithi038 ceph-mon[31227]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:30.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:34:30.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:30 smithi179 ceph-mon[35781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T05:34:31.463 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:31 smithi038 ceph-mon[31227]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:31 smithi047 ceph-mon[35327]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:31 smithi179 ceph-mon[35781]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:33.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:33 smithi038 ceph-mon[31227]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:33.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:34:33.574 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:33 smithi179 ceph-mon[35781]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:34:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:33 smithi047 ceph-mon[35327]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T05:34:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:34.524 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:34 smithi038 ceph-mon[31227]: Deploying daemon osd.3 on smithi038 2024-09-19T05:34:34.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:34 smithi179 ceph-mon[35781]: Deploying daemon osd.3 on smithi038 2024-09-19T05:34:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:34 smithi047 ceph-mon[35327]: Deploying daemon osd.3 on smithi038 2024-09-19T05:34:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:35 smithi038 ceph-mon[31227]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T05:34:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:35 smithi179 ceph-mon[35781]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T05:34:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:35 smithi047 ceph-mon[35327]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T05:34:37.600 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:37 smithi038 ceph-mon[31227]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T05:34:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:37 smithi179 ceph-mon[35781]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T05:34:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:37 smithi047 ceph-mon[35327]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T05:34:39.559 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:39 smithi038 ceph-mon[31227]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T05:34:39.559 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:39.559 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:39.559 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:39 smithi179 ceph-mon[35781]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T05:34:39.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:39.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:39.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:39 smithi047 ceph-mon[35327]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T05:34:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:41.461 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:41 smithi038 ceph-mon[31227]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:34:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:41 smithi179 ceph-mon[35781]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:34:41.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:41 smithi047 ceph-mon[35327]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:43.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:42 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.100+0000 7f99a97fa640 1 -- 172.21.15.38:0/1142425605 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f99ac062630 con 0x7f99940a31b0 2024-09-19T05:34:43.103 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 3 on host 'smithi038' 2024-09-19T05:34:43.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 msgr2=0x7f99940a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:43.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 0x7f99940a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f999c005fd0 tx=0x7f999c00b390 comp rx=0 tx=0).stop 2024-09-19T05:34:43.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 msgr2=0x7f99ac1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:43.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f99a0031610 tx=0x7f99a0004270 comp rx=0 tx=0).stop 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 shutdown_connections 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99940a31b0 0x7f99940a5670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99ac107960 0x7f99ac100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99ac105030 0x7f99ac1006b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 --2- 172.21.15.38:0/1142425605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ac102700 0x7f99ac102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 >> 172.21.15.38:0/1142425605 conn(0x7f99ac0fc280 msgr2=0x7f99ac103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:34:43.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 shutdown_connections 2024-09-19T05:34:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:34:43.102+0000 7f99b24d0640 1 -- 172.21.15.38:0/1142425605 wait complete. 2024-09-19T05:34:43.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:43.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:43.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:43.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:42 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:42 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:43.736 DEBUG:teuthology.orchestra.run.smithi038:osd.3> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T05:34:43.738 INFO:tasks.cephadm:Deploying osd.4 on smithi047 with /dev/vg_nvme/lv_3... 2024-09-19T05:34:43.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:34:43.999 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:34:44.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:44.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.045 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.120 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:44.120 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.120 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:34:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:44.894 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:34:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:34:44.406+0000 7f219b5c4640 -1 osd.3 0 log_to_monitors true 2024-09-19T05:34:45.228 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:45 smithi038 ceph-mon[31227]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:45.228 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:45 smithi038 ceph-mon[31227]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:34:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:45 smithi179 ceph-mon[35781]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:45 smithi179 ceph-mon[35781]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:34:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:45 smithi047 ceph-mon[35327]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T05:34:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:45 smithi047 ceph-mon[35327]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T05:34:46.348 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-mon[31227]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:34:46.349 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-mon[31227]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:34:46.349 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-mon[31227]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:34:46.349 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:46.358 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:46 smithi047 ceph-mon[35327]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:34:46.358 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:46 smithi047 ceph-mon[35327]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:34:46.358 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:46 smithi047 ceph-mon[35327]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:34:46.359 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:46.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:46 smithi179 ceph-mon[35781]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T05:34:46.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:46 smithi179 ceph-mon[35781]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T05:34:46.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:46 smithi179 ceph-mon[35781]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:34:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:34:46.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-09-19T05:34:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:34:47.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:34:46.825+0000 7f2197d5b640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T05:34:47.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:34:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:34:46.830+0000 7f218f367640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:34:47.363 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:47.363 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:34:47.363 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:34:47.363 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.363 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] boot 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:34:47.364 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:34:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] boot 2024-09-19T05:34:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:34:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.445+0000 7f63f42e5640 1 -- 172.21.15.47:0/191292154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec101c00 msgr2=0x7f63ec102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.445+0000 7f63f42e5640 1 --2- 172.21.15.47:0/191292154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec101c00 0x7f63ec102000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009900 tx=0x7f63e802a410 comp rx=0 tx=0).stop 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- 172.21.15.47:0/191292154 shutdown_connections 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- 172.21.15.47:0/191292154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 0x7f63ec10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- 172.21.15.47:0/191292154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec102e00 0x7f63ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- 172.21.15.47:0/191292154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec101c00 0x7f63ec102000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- 172.21.15.47:0/191292154 >> 172.21.15.47:0/191292154 conn(0x7f63ec0fd390 msgr2=0x7f63ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- 172.21.15.47:0/191292154 shutdown_connections 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- 172.21.15.47:0/191292154 wait complete. 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 Processor -- start 2024-09-19T05:34:47.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- start start 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec101c00 0x7f63ec197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 0x7f63ec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.446+0000 7f63f42e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10d8e0 con 0x7f63ec101c00 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f42e5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10d760 con 0x7f63ec102e00 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f42e5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10da60 con 0x7f63ec1037c0 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51142/0 (socket says 172.21.15.47:51142) 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 -- 172.21.15.47:0/2984435224 learned_addr learned my addr 172.21.15.47:0/2984435224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f2081640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec101c00 0x7f63ec197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f2882640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 0x7f63ec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:47.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 -- 172.21.15.47:0/2984435224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 msgr2=0x7f63ec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 0x7f63ec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 -- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec101c00 msgr2=0x7f63ec197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec101c00 0x7f63ec197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 -- 172.21.15.47:0/2984435224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec074420 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f1880640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c920 tx=0x7f63dc00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc007c70 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec074710 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63dc007e10 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc018890 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.447+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec074cd0 con 0x7f63ec102e00 2024-09-19T05:34:47.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.449+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec076110 con 0x7f63ec102e00 2024-09-19T05:34:47.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.450+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f63dc018a30 con 0x7f63ec102e00 2024-09-19T05:34:47.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.450+0000 7f63e37fe640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 0x7f63d80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:34:47.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.450+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f63dc0ce830 con 0x7f63ec102e00 2024-09-19T05:34:47.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.451+0000 7f63f2081640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 0x7f63d80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:34:47.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.451+0000 7f63f2081640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 0x7f63d80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005ab0 tx=0x7f63e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:34:47.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.455+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63dc014030 con 0x7f63ec102e00 2024-09-19T05:34:47.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:34:47.638+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f63ec108cd0 con 0x7f63d80a3270 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545]' entity='osd.3' 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: osd.3 [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] boot 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T05:34:47.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:34:48.398 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi038 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:48.399 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:48 smithi047 ceph-mon[35327]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:34:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:34:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi038 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:48.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:48 smithi179 ceph-mon[35781]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi038 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:34:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:34:48.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:48.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:48 smithi038 ceph-mon[31227]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T05:34:49.280 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:49 smithi047 ceph-mon[35327]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 441 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:49.280 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:49 smithi047 ceph-mon[35327]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:49.280 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:49 smithi047 ceph-mon[35327]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T05:34:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:49 smithi179 ceph-mon[35781]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 441 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:49 smithi179 ceph-mon[35781]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:49 smithi179 ceph-mon[35781]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T05:34:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:49 smithi038 ceph-mon[31227]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 441 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:49 smithi038 ceph-mon[31227]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:34:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:49 smithi038 ceph-mon[31227]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T05:34:50.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:50 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/260068815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:50 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:50 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]': finished 2024-09-19T05:34:50.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:50 smithi038 ceph-mon[31227]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:34:50.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:50.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:50 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/260068815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:50 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:50 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]': finished 2024-09-19T05:34:50.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:50 smithi047 ceph-mon[35327]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:34:50.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:50.570 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:50 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/260068815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.570 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:50 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]: dispatch 2024-09-19T05:34:50.570 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:50 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ea25802-a31e-4e01-bccf-183ee703afe8"}]': finished 2024-09-19T05:34:50.570 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:50 smithi179 ceph-mon[35781]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T05:34:50.570 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:51 smithi038 ceph-mon[31227]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:51 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/637418011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:51 smithi038 ceph-mon[31227]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:34:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:51 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:51.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:51 smithi047 ceph-mon[35327]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:51 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/637418011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:51 smithi047 ceph-mon[35327]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:34:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:51 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:51 smithi179 ceph-mon[35781]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:51 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/637418011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:34:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:51 smithi179 ceph-mon[35781]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T05:34:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:51 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:34:53.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:53 smithi047 ceph-mon[35327]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:53 smithi038 ceph-mon[31227]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:53 smithi179 ceph-mon[35781]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 440 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:34:55.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:55 smithi047 ceph-mon[35327]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T05:34:55.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:55 smithi038 ceph-mon[31227]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T05:34:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:55 smithi179 ceph-mon[35781]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-19T05:34:57.484 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:57 smithi047 ceph-mon[35327]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T05:34:57.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:57 smithi038 ceph-mon[31227]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T05:34:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:57 smithi179 ceph-mon[35781]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T05:34:59.580 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:59 smithi047 ceph-mon[35327]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:34:59.580 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:34:59.580 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:34:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:59 smithi038 ceph-mon[31227]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:34:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:34:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:34:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:34:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:59 smithi179 ceph-mon[35781]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:34:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T05:34:59.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:34:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:00.418 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:00 smithi047 ceph-mon[35327]: Deploying daemon osd.4 on smithi047 2024-09-19T05:35:00.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:00 smithi179 ceph-mon[35781]: Deploying daemon osd.4 on smithi047 2024-09-19T05:35:00.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:00 smithi038 ceph-mon[31227]: Deploying daemon osd.4 on smithi047 2024-09-19T05:35:01.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:01 smithi047 ceph-mon[35327]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:35:01.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:01 smithi038 ceph-mon[31227]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:35:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:01 smithi179 ceph-mon[35781]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:35:03.381 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:03 smithi047 ceph-mon[35327]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T05:35:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:03 smithi038 ceph-mon[31227]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T05:35:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:03 smithi179 ceph-mon[35781]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-19T05:35:05.268 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:05 smithi047 ceph-mon[35327]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:05.268 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:05.268 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:05.268 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:05 smithi038 ceph-mon[31227]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:05.700 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:05 smithi179 ceph-mon[35781]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:05.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:05.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:05.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:07.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:07.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:08.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.542+0000 7f63e37fe640 1 -- 172.21.15.47:0/2984435224 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f63ec108cd0 con 0x7f63d80a3270 2024-09-19T05:35:08.544 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 4 on host 'smithi047' 2024-09-19T05:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 msgr2=0x7f63d80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 0x7f63d80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005ab0 tx=0x7f63e8000bc0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 msgr2=0x7f63ec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c920 tx=0x7f63dc00cde0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 shutdown_connections 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63d80a3270 0x7f63d80a5730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec1037c0 0x7f63ec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec102e00 0x7f63ec198440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 --2- 172.21.15.47:0/2984435224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63ec101c00 0x7f63ec197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 >> 172.21.15.47:0/2984435224 conn(0x7f63ec0fd390 msgr2=0x7f63ec106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 shutdown_connections 2024-09-19T05:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:35:08.544+0000 7f63f42e5640 1 -- 172.21.15.47:0/2984435224 wait complete. 2024-09-19T05:35:09.134 DEBUG:teuthology.orchestra.run.smithi047:osd.4> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T05:35:09.136 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-09-19T05:35:09.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T05:35:09.389 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:35:09.436 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:09 smithi047 ceph-mon[35327]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:09.436 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:09.436 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:09.436 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:09 smithi179 ceph-mon[35781]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:09 smithi038 ceph-mon[31227]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:10.405 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:10 smithi047 ceph-mon[35327]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:10.405 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:10 smithi047 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:10.405 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:35:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:35:10.097+0000 7fb2a934e640 -1 osd.4 0 log_to_monitors true 2024-09-19T05:35:10.546 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:10 smithi179 ceph-mon[35781]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:10.546 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:10 smithi179 ceph-mon[35781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:10 smithi038 ceph-mon[31227]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:10 smithi038 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T05:35:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:11.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:35:11.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:35:11.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:11.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:11 smithi179 ceph-mon[35781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.590 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:35:11.297+0000 7fb2a52d1640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T05:35:11.590 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:35:11.302+0000 7fb2a10f9640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:11.591 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:11 smithi047 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: from='osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:11 smithi038 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:35:11.922 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:35:11.950 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-09-19T05:35:12.196 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:35:12.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 -- 172.21.15.179:0/3819140290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 msgr2=0x7f0bac103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 --2- 172.21.15.179:0/3819140290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac103280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009900 tx=0x7f0ba802a410 comp rx=0 tx=0).stop 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 -- 172.21.15.179:0/3819140290 shutdown_connections 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 --2- 172.21.15.179:0/3819140290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 --2- 172.21.15.179:0/3819140290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac103280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 --2- 172.21.15.179:0/3819140290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 0x7f0bac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 -- 172.21.15.179:0/3819140290 >> 172.21.15.179:0/3819140290 conn(0x7f0bac0fd390 msgr2=0x7f0bac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 -- 172.21.15.179:0/3819140290 shutdown_connections 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.653+0000 7f0bb3570640 1 -- 172.21.15.179:0/3819140290 wait complete. 2024-09-19T05:35:12.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 Processor -- start 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 -- start start 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 0x7f0bac197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10d8e0 con 0x7f0bac101c00 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10d760 con 0x7f0bac1037c0 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb3570640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10da60 con 0x7f0bac102e00 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:12.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:52934/0 (socket says 172.21.15.179:52934) 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 -- 172.21.15.179:0/95016237 learned_addr learned my addr 172.21.15.179:0/95016237 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb130c640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 0x7f0bac197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb1b0d640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 -- 172.21.15.179:0/95016237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 msgr2=0x7f0bac19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 -- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 msgr2=0x7f0bac197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 0x7f0bac197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 -- 172.21.15.179:0/95016237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac19fb00 con 0x7f0bac102e00 2024-09-19T05:35:12.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb1b0d640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.654+0000 7f0bb0b0b640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009230 tx=0x7f0ba8004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.655+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba803f030 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.655+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bac19fd90 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.655+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba8003ce0 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.655+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8035070 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.655+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bac1a02e0 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.656+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bac108f50 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.657+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0ba8042030 con 0x7f0bac102e00 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.657+0000 7f0ba27fc640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 0x7f0b980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:12.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.658+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f0ba80eefd0 con 0x7f0bac102e00 2024-09-19T05:35:12.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.658+0000 7f0bb130c640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 0x7f0b980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:12.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.658+0000 7f0bb130c640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 0x7f0b980a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0bac102c60 tx=0x7f0b9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:35:12.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.662+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ba80bcd60 con 0x7f0bac102e00 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:12.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:12.840 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:12.837+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0bac062630 con 0x7f0b980a3270 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi047 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] boot 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:35:13.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:13.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:13.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:13.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi047 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-19T05:35:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] boot 2024-09-19T05:35:13.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:35:13.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:13.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:13.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:13.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi047 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-19T05:35:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: osd.4 [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] boot 2024-09-19T05:35:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T05:35:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:35:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:13.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:14.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:14 smithi179 ceph-mon[35781]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:14.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:14 smithi179 ceph-mon[35781]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:35:14.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:14 smithi038 ceph-mon[31227]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:14.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:14 smithi038 ceph-mon[31227]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:35:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:14 smithi047 ceph-mon[35327]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:14.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:14 smithi047 ceph-mon[35327]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/1319171786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]': finished 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:35:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/1319171786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]': finished 2024-09-19T05:35:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:35:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/1319171786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]: dispatch 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03dfe06a-5da0-4925-a09a-9fea434b0633"}]': finished 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T05:35:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:16.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:16 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/202033020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:16.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:16 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/202033020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:16.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:16 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/202033020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:17.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:17 smithi038 ceph-mon[31227]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:17 smithi179 ceph-mon[35781]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:17 smithi047 ceph-mon[35327]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:19.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:19 smithi038 ceph-mon[31227]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:19 smithi179 ceph-mon[35781]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:19 smithi047 ceph-mon[35327]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:21.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:21 smithi038 ceph-mon[31227]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:21.678 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:21 smithi179 ceph-mon[35781]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:21 smithi047 ceph-mon[35327]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:23.599 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:23 smithi179 ceph-mon[35781]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:23.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:23 smithi038 ceph-mon[31227]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:23 smithi047 ceph-mon[35327]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:35:24.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:24.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:35:24.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:24.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T05:35:24.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:25.576 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:25 smithi179 ceph-mon[35781]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:25.577 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:25 smithi179 ceph-mon[35781]: Deploying daemon osd.5 on smithi179 2024-09-19T05:35:25.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:25 smithi038 ceph-mon[31227]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:25 smithi038 ceph-mon[31227]: Deploying daemon osd.5 on smithi179 2024-09-19T05:35:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:25 smithi047 ceph-mon[35327]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:25 smithi047 ceph-mon[35327]: Deploying daemon osd.5 on smithi179 2024-09-19T05:35:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:27 smithi038 ceph-mon[31227]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:27.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:27 smithi179 ceph-mon[35781]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:27.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:27 smithi047 ceph-mon[35327]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:29.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:29 smithi179 ceph-mon[35781]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:29.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:29 smithi038 ceph-mon[31227]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:29.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:29 smithi047 ceph-mon[35327]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:30.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:30.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:30.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:30 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:30.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:30.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:30.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:30 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:30.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:30.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:30.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:30 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:31.640 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:31 smithi179 ceph-mon[35781]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:31.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:31 smithi047 ceph-mon[35327]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:31 smithi038 ceph-mon[31227]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:33.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:33.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:33.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:33.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:33.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.931+0000 7f0ba27fc640 1 -- 172.21.15.179:0/95016237 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bac062630 con 0x7f0b980a3270 2024-09-19T05:35:33.933 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-09-19T05:35:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 msgr2=0x7f0b980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 0x7f0b980a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0bac102c60 tx=0x7f0b9c005f90 comp rx=0 tx=0).stop 2024-09-19T05:35:33.935 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 msgr2=0x7f0bac198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009230 tx=0x7f0ba8004430 comp rx=0 tx=0).stop 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 shutdown_connections 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b980a3270 0x7f0b980a5730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bac1037c0 0x7f0bac19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bac102e00 0x7f0bac198410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 --2- 172.21.15.179:0/95016237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0bac101c00 0x7f0bac197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.933+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 >> 172.21.15.179:0/95016237 conn(0x7f0bac0fd390 msgr2=0x7f0bac106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.934+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 shutdown_connections 2024-09-19T05:35:33.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:35:33.934+0000 7f0bb3570640 1 -- 172.21.15.179:0/95016237 wait complete. 2024-09-19T05:35:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.634 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T05:35:34.637 INFO:tasks.cephadm:Deploying osd.6 on smithi038 with /dev/vg_nvme/lv_2... 2024-09-19T05:35:34.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:35:34.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:34.902 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:35:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:35 smithi038 ceph-mon[31227]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:35 smithi038 ceph-mon[31227]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:35 smithi038 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:35 smithi179 ceph-mon[35781]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:35 smithi179 ceph-mon[35781]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:35 smithi179 ceph-mon[35781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:35.698 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:35:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:35:35.206+0000 7f9792019640 -1 osd.5 0 log_to_monitors true 2024-09-19T05:35:35.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:35 smithi047 ceph-mon[35327]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:35.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:35 smithi047 ceph-mon[35327]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:35 smithi047 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T05:35:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:36 smithi038 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:35:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:36 smithi038 ceph-mon[31227]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:35:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:36 smithi038 ceph-mon[31227]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:36 smithi038 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-mon[35781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:35:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-mon[35781]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:35:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-mon[35781]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-mon[35781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:36.698 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:35:36.356+0000 7f978df9c640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T05:35:36.698 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:35:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:35:36.365+0000 7f9789dc4640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:35:36.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:36 smithi047 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T05:35:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:36 smithi047 ceph-mon[35327]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T05:35:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:36 smithi047 ceph-mon[35327]: from='osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:36 smithi047 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:35:37.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:37.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:35:37.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:35:37.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] boot 2024-09-19T05:35:37.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:35:37.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:35:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:35:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] boot 2024-09-19T05:35:37.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:35:37.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-19T05:35:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:35:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T05:35:37.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: osd.5 [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] boot 2024-09-19T05:35:37.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T05:35:37.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:35:37.803 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:35:37.829 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-09-19T05:35:38.130 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi179 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi179 to 8544M 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:38.571 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:38 smithi038 ceph-mon[31227]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:35:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 -- 172.21.15.38:0/4239075703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420101cd0 msgr2=0x7f0420102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 --2- 172.21.15.38:0/4239075703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420101cd0 0x7f0420102150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0410009900 tx=0x7f041002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 -- 172.21.15.38:0/4239075703 shutdown_connections 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 --2- 172.21.15.38:0/4239075703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420102690 0x7f0420108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 --2- 172.21.15.38:0/4239075703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420101cd0 0x7f0420102150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 --2- 172.21.15.38:0/4239075703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0420100ad0 0x7f0420100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 -- 172.21.15.38:0/4239075703 >> 172.21.15.38:0/4239075703 conn(0x7f04200fc280 msgr2=0x7f04200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.580+0000 7f0427f8e640 1 -- 172.21.15.38:0/4239075703 shutdown_connections 2024-09-19T05:35:38.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 -- 172.21.15.38:0/4239075703 wait complete. 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 Processor -- start 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 -- start start 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420100ad0 0x7f0420197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0420101cd0 0x7f0420198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.581+0000 7f0427f8e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f042010d8e0 con 0x7f0420102690 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0427f8e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f042010d760 con 0x7f0420100ad0 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0427f8e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f042010da60 con 0x7f0420101cd0 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41508/0 (socket says 172.21.15.38:41508) 2024-09-19T05:35:38.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 -- 172.21.15.38:0/611532103 learned_addr learned my addr 172.21.15.38:0/611532103 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0425d2a640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420100ad0 0x7f0420197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 -- 172.21.15.38:0/611532103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0420101cd0 msgr2=0x7f0420198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0420101cd0 0x7f0420198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 -- 172.21.15.38:0/611532103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420100ad0 msgr2=0x7f0420197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420100ad0 0x7f0420197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 -- 172.21.15.38:0/611532103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0420109960 con 0x7f0420102690 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f042652b640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f041800cce0 tx=0x7f041800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041800f450 con 0x7f0420102690 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0420109c50 con 0x7f0420102690 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041800fad0 con 0x7f0420102690 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.582+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418018b60 con 0x7f0420102690 2024-09-19T05:35:38.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.583+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042010a1c0 con 0x7f0420102690 2024-09-19T05:35:38.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.584+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042010b660 con 0x7f0420102690 2024-09-19T05:35:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.585+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f041800fc80 con 0x7f0420102690 2024-09-19T05:35:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.585+0000 7f0416ffd640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 0x7f040c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:35:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.586+0000 7f0425d2a640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 0x7f040c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:35:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.586+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7f04180cf190 con 0x7f0420102690 2024-09-19T05:35:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.586+0000 7f0425d2a640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 0x7f040c0a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0420101b30 tx=0x7f041c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:35:38.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.589+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0418014030 con 0x7f0420102690 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi179 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi179 to 8544M 2024-09-19T05:35:38.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:38.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:38.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:38 smithi047 ceph-mon[35327]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:35:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:38.778+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0420107d80 con 0x7f040c0a3270 2024-09-19T05:35:38.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:35:38.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi179 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi179 to 8544M 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:38.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:38 smithi179 ceph-mon[35781]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T05:35:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:39 smithi038 ceph-mon[31227]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:39 smithi038 ceph-mon[31227]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:35:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:39 smithi179 ceph-mon[35781]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:39.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:39.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:39 smithi179 ceph-mon[35781]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:39 smithi047 ceph-mon[35327]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:39 smithi047 ceph-mon[35327]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T05:35:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:40 smithi038 ceph-mon[31227]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:40 smithi179 ceph-mon[35781]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:40.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:40 smithi047 ceph-mon[35327]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]: dispatch 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]': finished 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:35:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:41 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:35:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T05:35:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]: dispatch 2024-09-19T05:35:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]': finished 2024-09-19T05:35:41.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:35:41.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:41 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]: dispatch 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2056614964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e40f682d-ec98-470b-b2e2-19946849a6f0"}]': finished 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T05:35:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:41 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:35:42.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:42 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2749829877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:42.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:42 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2749829877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:42.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:42 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2749829877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:35:43.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:43 smithi038 ceph-mon[31227]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:43 smithi179 ceph-mon[35781]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:43.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:43 smithi047 ceph-mon[35327]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:45 smithi038 ceph-mon[31227]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:35:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:45 smithi179 ceph-mon[35781]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:35:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:45 smithi047 ceph-mon[35327]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T05:35:46.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:46 smithi038 ceph-mon[31227]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T05:35:46.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:46 smithi179 ceph-mon[35781]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T05:35:46.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:46 smithi047 ceph-mon[35327]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T05:35:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:48 smithi038 ceph-mon[31227]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:35:48.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:48 smithi179 ceph-mon[35781]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:35:48.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:48 smithi047 ceph-mon[35327]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:35:50.830 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:50 smithi038 ceph-mon[31227]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:35:50.831 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:35:50.831 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:50.831 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:50 smithi038 ceph-mon[31227]: Deploying daemon osd.6 on smithi038 2024-09-19T05:35:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:50 smithi179 ceph-mon[35781]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:35:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:35:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:50 smithi179 ceph-mon[35781]: Deploying daemon osd.6 on smithi038 2024-09-19T05:35:50.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:50 smithi047 ceph-mon[35327]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:35:50.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T05:35:50.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:50.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:50 smithi047 ceph-mon[35327]: Deploying daemon osd.6 on smithi038 2024-09-19T05:35:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:52 smithi038 ceph-mon[31227]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T05:35:52.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:52 smithi179 ceph-mon[35781]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T05:35:52.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:52 smithi047 ceph-mon[35327]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T05:35:54.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:54 smithi038 ceph-mon[31227]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:54.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:54 smithi179 ceph-mon[35781]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:54.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:54 smithi047 ceph-mon[35327]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:35:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:56 smithi038 ceph-mon[31227]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:56 smithi179 ceph-mon[35781]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:56 smithi047 ceph-mon[35327]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:35:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:58.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:58 smithi038 ceph-mon[31227]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:58.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:58 smithi179 ceph-mon[35781]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:58.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:58 smithi047 ceph-mon[35327]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:35:59.647 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 6 on host 'smithi038' 2024-09-19T05:35:59.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.645+0000 7f0416ffd640 1 -- 172.21.15.38:0/611532103 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0420107d80 con 0x7f040c0a3270 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 msgr2=0x7f040c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 0x7f040c0a5730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0420101b30 tx=0x7f041c009110 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 msgr2=0x7f042019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f041800cce0 tx=0x7f041800a620 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 shutdown_connections 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f040c0a3270 0x7f040c0a5730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0420102690 0x7f042019f4f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0420101cd0 0x7f0420198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 --2- 172.21.15.38:0/611532103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0420100ad0 0x7f0420197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 >> 172.21.15.38:0/611532103 conn(0x7f04200fc280 msgr2=0x7f0420105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 shutdown_connections 2024-09-19T05:35:59.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:35:59.648+0000 7f0427f8e640 1 -- 172.21.15.38:0/611532103 wait complete. 2024-09-19T05:35:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:35:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:35:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:35:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:35:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:35:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:35:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.322 DEBUG:teuthology.orchestra.run.smithi038:osd.6> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T05:36:00.324 INFO:tasks.cephadm:Deploying osd.7 on smithi047 with /dev/vg_nvme/lv_2... 2024-09-19T05:36:00.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:36:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:36:00.797 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:00 smithi038 ceph-mon[31227]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:00.797 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:00.797 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.797 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.917 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:00 smithi047 ceph-mon[35327]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:00.918 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:00.918 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.918 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:00 smithi179 ceph-mon[35781]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:01.831 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:01 smithi038 ceph-mon[31227]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:36:01.832 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:36:01 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:36:01.514+0000 7ff75a413640 -1 osd.6 0 log_to_monitors true 2024-09-19T05:36:01.930 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:01 smithi047 ceph-mon[35327]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:36:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:01 smithi179 ceph-mon[35781]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T05:36:02.829 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:02 smithi047 ceph-mon[35327]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:02.829 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:02 smithi047 ceph-mon[35327]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:36:02.829 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:02 smithi047 ceph-mon[35327]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:36:02.829 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:02 smithi047 ceph-mon[35327]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:36:02.829 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:02.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:02 smithi038 ceph-mon[31227]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:02.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:02 smithi038 ceph-mon[31227]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:36:02.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:02 smithi038 ceph-mon[31227]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:36:02.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:02 smithi038 ceph-mon[31227]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:36:02.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:02.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:02 smithi179 ceph-mon[35781]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:02.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:02 smithi179 ceph-mon[35781]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T05:36:02.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:02 smithi179 ceph-mon[35781]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T05:36:02.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:02 smithi179 ceph-mon[35781]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:36:02.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:36:03.144 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-09-19T05:36:03.397 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:36:03.644 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:36:03.221+0000 7ff756baa640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T05:36:03.644 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:36:03.229+0000 7ff74e1b6640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:36:03.783 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:36:03.783 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:36:03.783 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.783 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi038 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.784 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:03 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 -- 172.21.15.47:0/250651561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8100490 msgr2=0x7f3ae8100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 --2- 172.21.15.47:0/250651561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8100490 0x7f3ae8100890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009900 tx=0x7f3ae402a410 comp rx=0 tx=0).stop 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 -- 172.21.15.47:0/250651561 shutdown_connections 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 --2- 172.21.15.47:0/250651561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8101770 0x7f3ae8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 --2- 172.21.15.47:0/250651561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 0x7f3ae8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 --2- 172.21.15.47:0/250651561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8100490 0x7f3ae8100890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 -- 172.21.15.47:0/250651561 >> 172.21.15.47:0/250651561 conn(0x7f3ae80fc2a0 msgr2=0x7f3ae80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 -- 172.21.15.47:0/250651561 shutdown_connections 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.865+0000 7f3aef29d640 1 -- 172.21.15.47:0/250651561 wait complete. 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 Processor -- start 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 -- start start 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8100490 0x7f3ae8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 0x7f3ae819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810fa60 con 0x7f3ae8100dd0 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810f8e0 con 0x7f3ae8101770 2024-09-19T05:36:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aef29d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810fbe0 con 0x7f3ae8100490 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aed83a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aed83a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55184/0 (socket says 172.21.15.47:55184) 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aed83a640 1 -- 172.21.15.47:0/1567805695 learned_addr learned my addr 172.21.15.47:0/1567805695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.866+0000 7f3aed039640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8100490 0x7f3ae8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aec838640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 0x7f3ae819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 -- 172.21.15.47:0/1567805695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8100490 msgr2=0x7f3ae8199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8100490 0x7f3ae8199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 -- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 msgr2=0x7f3ae819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 0x7f3ae819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:03.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 -- 172.21.15.47:0/1567805695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae81a1c20 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aed83a640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000cce0 tx=0x7f3ae000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000f450 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae000fad0 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae81a1f10 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0018c10 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.867+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81a23c0 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.868+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae810b1e0 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.869+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3ae000f5f0 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.870+0000 7f3ade7fc640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 0x7f3ad40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.870+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f3ae00cf1d0 con 0x7f3ae8101770 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.870+0000 7f3aed039640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 0x7f3ad40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.871+0000 7f3aed039640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 0x7f3ad40a57b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40314e0 tx=0x7f3ae4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:03.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:03.873+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ae0014030 con 0x7f3ae8101770 2024-09-19T05:36:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:36:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi038 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T05:36:03.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:03.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:03.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:03.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:03 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:04.057+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3ae8062630 con 0x7f3ad40a32f0 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184]' entity='osd.6' 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi038 2024-09-19T05:36:04.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:04.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:03 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] boot 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:36:04.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:04.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:36:04.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:36:04.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:04.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] boot 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:36:04.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:36:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:36:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-19T05:36:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:05.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:05.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:05.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: osd.6 [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] boot 2024-09-19T05:36:05.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T05:36:05.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:36:06.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:36:06.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/1786978769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]': finished 2024-09-19T05:36:06.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:36:06.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/1786978769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]': finished 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:36:06.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T05:36:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/1786978769' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]: dispatch 2024-09-19T05:36:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a99cd83f-58cb-47b9-bcea-35e82ceacbc8"}]': finished 2024-09-19T05:36:06.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T05:36:06.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:07 smithi179 ceph-mon[35781]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:07 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/166751497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:07 smithi047 ceph-mon[35327]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:07 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/166751497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:07 smithi038 ceph-mon[31227]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:07 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/166751497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:09.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:09 smithi038 ceph-mon[31227]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:09 smithi179 ceph-mon[35781]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:09 smithi047 ceph-mon[35327]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:11 smithi179 ceph-mon[35781]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:11.698 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:11 smithi047 ceph-mon[35327]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:11 smithi038 ceph-mon[31227]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:13.575 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:13 smithi047 ceph-mon[35327]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:13 smithi179 ceph-mon[35781]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:13 smithi038 ceph-mon[31227]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:15 smithi047 ceph-mon[35327]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:36:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:15 smithi038 ceph-mon[31227]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:36:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:15 smithi179 ceph-mon[35781]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T05:36:15.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:16.586 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:16 smithi047 ceph-mon[35327]: Deploying daemon osd.7 on smithi047 2024-09-19T05:36:16.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:16 smithi038 ceph-mon[31227]: Deploying daemon osd.7 on smithi047 2024-09-19T05:36:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:16 smithi179 ceph-mon[35781]: Deploying daemon osd.7 on smithi047 2024-09-19T05:36:17.682 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:17 smithi047 ceph-mon[35327]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:17 smithi038 ceph-mon[31227]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:17 smithi179 ceph-mon[35781]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:19 smithi047 ceph-mon[35327]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:19.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:19 smithi038 ceph-mon[31227]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:19 smithi179 ceph-mon[35781]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:21.669 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:21 smithi047 ceph-mon[35327]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:21.670 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:21.670 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:21.670 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:21.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:21 smithi179 ceph-mon[35781]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:21.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:21.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:21.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:21 smithi038 ceph-mon[31227]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:23.686 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:23 smithi047 ceph-mon[35327]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:23.728 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 7 on host 'smithi047' 2024-09-19T05:36:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.726+0000 7f3ade7fc640 1 -- 172.21.15.47:0/1567805695 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ae8062630 con 0x7f3ad40a32f0 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 msgr2=0x7f3ad40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 0x7f3ad40a57b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40314e0 tx=0x7f3ae4000bc0 comp rx=0 tx=0).stop 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 msgr2=0x7f3ae81a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000cce0 tx=0x7f3ae000a620 comp rx=0 tx=0).stop 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 shutdown_connections 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3ad40a32f0 0x7f3ad40a57b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae8101770 0x7f3ae81a15b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae8100dd0 0x7f3ae819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 --2- 172.21.15.47:0/1567805695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae8100490 0x7f3ae8199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 >> 172.21.15.47:0/1567805695 conn(0x7f3ae80fc2a0 msgr2=0x7f3ae80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:36:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 shutdown_connections 2024-09-19T05:36:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:36:23.728+0000 7f3aef29d640 1 -- 172.21.15.47:0/1567805695 wait complete. 2024-09-19T05:36:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:23 smithi038 ceph-mon[31227]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:23 smithi179 ceph-mon[35781]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:24.400 DEBUG:teuthology.orchestra.run.smithi047:osd.7> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T05:36:24.403 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-09-19T05:36:24.403 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T05:36:24.652 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:36:24.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:24 smithi047 ceph-mon[35327]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:24.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:24.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:24.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:24.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.017 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:24 smithi179 ceph-mon[35781]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:25.017 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.017 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.018 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.018 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:24 smithi038 ceph-mon[31227]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:26.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:26 smithi179 ceph-mon[35781]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:26.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:26 smithi047 ceph-mon[35327]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:26.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:36:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:36:26.736+0000 7fc77bf61640 -1 osd.7 0 log_to_monitors true 2024-09-19T05:36:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:26 smithi038 ceph-mon[31227]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:27.362 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:36:27.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-09-19T05:36:27.640 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:36:28.029 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi047 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.030 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.031 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:28.031 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.031 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.031 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 -- 172.21.15.179:0/825428090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8100ad0 msgr2=0x7f92f8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 --2- 172.21.15.179:0/825428090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8100ad0 0x7f92f8100ed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009900 tx=0x7f92ec02a410 comp rx=0 tx=0).stop 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 -- 172.21.15.179:0/825428090 shutdown_connections 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 --2- 172.21.15.179:0/825428090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8102690 0x7f92f8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 --2- 172.21.15.179:0/825428090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8101cd0 0x7f92f8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 --2- 172.21.15.179:0/825428090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8100ad0 0x7f92f8100ed0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 -- 172.21.15.179:0/825428090 >> 172.21.15.179:0/825428090 conn(0x7f92f80fc280 msgr2=0x7f92f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 -- 172.21.15.179:0/825428090 shutdown_connections 2024-09-19T05:36:28.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.110+0000 7f92fef79640 1 -- 172.21.15.179:0/825428090 wait complete. 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 Processor -- start 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 -- start start 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8100ad0 0x7f92f8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8102690 0x7f92f819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d880 con 0x7f92f8100ad0 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810d700 con 0x7f92f8102690 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fef79640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f92f810da00 con 0x7f92f8101cd0 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:28.139 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47678/0 (socket says 172.21.15.179:47678) 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 -- 172.21.15.179:0/2394990317 learned_addr learned my addr 172.21.15.179:0/2394990317 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fcd15640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8100ad0 0x7f92f8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92fd516640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8102690 0x7f92f819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 -- 172.21.15.179:0/2394990317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8102690 msgr2=0x7f92f819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8102690 0x7f92f819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 -- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8100ad0 msgr2=0x7f92f8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8100ad0 0x7f92f8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.111+0000 7f92f7fff640 1 -- 172.21.15.179:0/2394990317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f819fa90 con 0x7f92f8101cd0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92f7fff640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c920 tx=0x7f92e400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4007c70 con 0x7f92f8101cd0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e4007e10 con 0x7f92f8101cd0 2024-09-19T05:36:28.140 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f819fd80 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4018890 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.112+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f81a0240 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.113+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f8107e20 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.114+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f92e4018a30 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.115+0000 7f92f5ffb640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 0x7f92e80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.115+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5101+0+0 (secure 0 0 0) 0x7f92e40ced30 con 0x7f92f8101cd0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.115+0000 7f92fcd15640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 0x7f92e80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.115+0000 7f92fcd15640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 0x7f92e80a57b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f92ec005ab0 tx=0x7f92ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:28.141 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.119+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92e4014030 con 0x7f92f8101cd0 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi047 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:28.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi047 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:28.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:28.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:28.296+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f92f8062630 con 0x7f92e80a32f0 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:36:28.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:28.976 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:28.976 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:28.976 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:28.976 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:28.976 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:28 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:29.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:28 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:29.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:36:28.744+0000 7fc777ee4640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T05:36:29.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:36:28.754+0000 7fc773d0c640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:36:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-19T05:36:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:29.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:29.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:28 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:29.958 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] boot 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:36:29.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:29 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] boot 2024-09-19T05:36:30.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:36:30.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:29 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: osd.7 [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] boot 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T05:36:30.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:29 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/4045773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]': finished 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:36:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/4045773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]': finished 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:36:31.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/4045773134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]: dispatch 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1306f615-40c3-4be2-b882-b665b1b01588"}]': finished 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T05:36:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:32.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:32 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/2447015200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:32.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:32 smithi047 ceph-mon[35327]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:36:32.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:32 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:32.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:32 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/2447015200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:32.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:32 smithi038 ceph-mon[31227]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:36:32.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:32 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:32.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:32 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/2447015200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:32 smithi179 ceph-mon[35781]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T05:36:32.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:32 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:33 smithi038 ceph-mon[31227]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:33 smithi038 ceph-mon[31227]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:36:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:33 smithi179 ceph-mon[35781]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:33 smithi179 ceph-mon[35781]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:36:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:33.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:33 smithi047 ceph-mon[35327]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:33.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:33 smithi047 ceph-mon[35327]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T05:36:33.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:35 smithi038 ceph-mon[31227]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:35 smithi179 ceph-mon[35781]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:35.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:35 smithi047 ceph-mon[35327]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:37.330 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:37 smithi179 ceph-mon[35781]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:37 smithi038 ceph-mon[31227]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:37.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:37 smithi047 ceph-mon[35327]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:39.387 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:39 smithi179 ceph-mon[35781]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:39.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:39 smithi038 ceph-mon[31227]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:39.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:39 smithi047 ceph-mon[35327]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:40.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:36:40.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:40.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:40 smithi038 ceph-mon[31227]: Deploying daemon osd.8 on smithi179 2024-09-19T05:36:40.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:36:40.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:40.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:40 smithi179 ceph-mon[35781]: Deploying daemon osd.8 on smithi179 2024-09-19T05:36:40.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T05:36:40.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:40.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:40 smithi047 ceph-mon[35327]: Deploying daemon osd.8 on smithi179 2024-09-19T05:36:41.375 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:41 smithi179 ceph-mon[35781]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:41.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:41 smithi038 ceph-mon[31227]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:41.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:41 smithi047 ceph-mon[35327]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:43.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:43 smithi038 ceph-mon[31227]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:43 smithi179 ceph-mon[35781]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:43 smithi047 ceph-mon[35327]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:45.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:45 smithi179 ceph-mon[35781]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:45.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:45 smithi038 ceph-mon[31227]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:45 smithi047 ceph-mon[35327]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:46.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:46.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:46.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:46.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:46.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:46.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:47.861 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:47 smithi179 ceph-mon[35781]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:47 smithi038 ceph-mon[31227]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:47 smithi047 ceph-mon[35327]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:49.385 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.428+0000 7f92f5ffb640 1 -- 172.21.15.179:0/2394990317 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92f8062630 con 0x7f92e80a32f0 2024-09-19T05:36:49.431 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 msgr2=0x7f92e80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 0x7f92e80a57b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f92ec005ab0 tx=0x7f92ec000bc0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 msgr2=0x7f92f81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c920 tx=0x7f92e400cde0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 shutdown_connections 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92e80a32f0 0x7f92e80a57b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92f8102690 0x7f92f819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92f8101cd0 0x7f92f81983a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 --2- 172.21.15.179:0/2394990317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f8100ad0 0x7f92f8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:49.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 >> 172.21.15.179:0/2394990317 conn(0x7f92f80fc280 msgr2=0x7f92f8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:36:49.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 shutdown_connections 2024-09-19T05:36:49.436 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:36:49.431+0000 7f92fef79640 1 -- 172.21.15.179:0/2394990317 wait complete. 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.118 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T05:36:50.120 INFO:tasks.cephadm:Deploying osd.9 on smithi038 with /dev/vg_nvme/lv_1... 2024-09-19T05:36:50.120 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:36:50.388 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:36:50.408 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:50.408 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.408 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:50.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.627 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:36:50.628 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:50.628 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:51.435 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:36:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:36:51.309+0000 7f1dd45bf640 -1 osd.8 0 log_to_monitors true 2024-09-19T05:36:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:51 smithi179 ceph-mon[35781]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:51 smithi179 ceph-mon[35781]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:51 smithi179 ceph-mon[35781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:51 smithi047 ceph-mon[35327]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:51 smithi047 ceph-mon[35327]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:51 smithi047 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:51.769 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:51 smithi038 ceph-mon[31227]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:51.770 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:51 smithi038 ceph-mon[31227]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:51.770 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:51 smithi038 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:36:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:52 smithi038 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:36:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:52 smithi038 ceph-mon[31227]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:36:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:52 smithi038 ceph-mon[31227]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:52.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:52 smithi038 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:52.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:52 smithi047 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:36:52.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:52 smithi047 ceph-mon[35327]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:36:52.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:52 smithi047 ceph-mon[35327]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:52.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:52.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:52 smithi047 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:52.866 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:52 smithi179 ceph-mon[35781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:36:52.867 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:52 smithi179 ceph-mon[35781]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:36:52.867 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:52 smithi179 ceph-mon[35781]: from='osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:52.867 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:52.867 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:52 smithi179 ceph-mon[35781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:36:53.184 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:36:53.210 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-09-19T05:36:53.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:36:53.155+0000 7f1dd0542640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T05:36:53.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:36:53.166+0000 7f1dc8362640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:36:53.476 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='osd.8 ' entity='osd.8' 2024-09-19T05:36:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:53.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:53 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='osd.8 ' entity='osd.8' 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.743 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:53.744 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:53 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:36:53.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='osd.8 ' entity='osd.8' 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:53.874 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:36:53.875 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:53 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 -- 172.21.15.38:0/1601807820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 msgr2=0x7f431c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 --2- 172.21.15.38:0/1601807820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c10a050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4314009ca0 tx=0x7f431402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 -- 172.21.15.38:0/1601807820 shutdown_connections 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 --2- 172.21.15.38:0/1601807820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c10a050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 --2- 172.21.15.38:0/1601807820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c102e00 0x7f431c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 --2- 172.21.15.38:0/1601807820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c101c00 0x7f431c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 -- 172.21.15.38:0/1601807820 >> 172.21.15.38:0/1601807820 conn(0x7f431c0fd390 msgr2=0x7f431c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 -- 172.21.15.38:0/1601807820 shutdown_connections 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.970+0000 7f43229ab640 1 -- 172.21.15.38:0/1601807820 wait complete. 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 Processor -- start 2024-09-19T05:36:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 -- start start 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c101c00 0x7f431c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 0x7f431c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10d900 con 0x7f431c1037c0 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10d780 con 0x7f431c101c00 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.971+0000 7f43229ab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10da80 con 0x7f431c102e00 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50624/0 (socket says 172.21.15.38:50624) 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 -- 172.21.15.38:0/2892981056 learned_addr learned my addr 172.21.15.38:0/2892981056 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f431bfff640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c101c00 0x7f431c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f431b7fe640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 0x7f431c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 -- 172.21.15.38:0/2892981056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 msgr2=0x7f431c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 0x7f431c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 -- 172.21.15.38:0/2892981056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c101c00 msgr2=0x7f431c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c101c00 0x7f431c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 -- 172.21.15.38:0/2892981056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c19fb50 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f431b7fe640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 0x7f431c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f4320f48640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4314005e30 tx=0x7f4314031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314003c30 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4314003dd0 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431402fde0 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c19fde0 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.972+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a0310 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.973+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c109000 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.974+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4314004050 con 0x7f431c1037c0 2024-09-19T05:36:53.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.975+0000 7f43197fa640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 0x7f43040a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:36:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.976+0000 7f431bfff640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 0x7f43040a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:36:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.976+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5862+0+0 (secure 0 0 0) 0x7f43140f0250 con 0x7f431c1037c0 2024-09-19T05:36:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.976+0000 7f431bfff640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 0x7f43040a5730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f430c005fd0 tx=0x7f430c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:36:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:53.980+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43140bd860 con 0x7f431c1037c0 2024-09-19T05:36:54.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:36:54.166+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f431c062630 con 0x7f43040a3270 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi179 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi179 to 5696M 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] boot 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:54.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:54.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi179 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi179 to 5696M 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] boot 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:36:54.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:54.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:54.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:54.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:36:54.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi179 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi179 to 5696M 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: osd.8 [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] boot 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:36:54.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:36:54.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:36:54.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:36:55.598 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:55 smithi038 ceph-mon[31227]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:55.598 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:55 smithi038 ceph-mon[31227]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:55.598 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:55 smithi038 ceph-mon[31227]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:36:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:55 smithi047 ceph-mon[35327]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:55 smithi047 ceph-mon[35327]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:55 smithi047 ceph-mon[35327]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:36:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:55 smithi179 ceph-mon[35781]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:55 smithi179 ceph-mon[35781]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:36:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:55 smithi179 ceph-mon[35781]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]: dispatch 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]': finished 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:36:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:36:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:36:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]: dispatch 2024-09-19T05:36:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]': finished 2024-09-19T05:36:57.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:36:57.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]: dispatch 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1073179365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3874c68-e48a-452f-a6f4-c349235e29da"}]': finished 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T05:36:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:36:58.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:58 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/586177671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:58.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:58 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/586177671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:58.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:58 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/586177671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:36:59.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:36:59 smithi038 ceph-mon[31227]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:59.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:36:59 smithi179 ceph-mon[35781]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:36:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:36:59 smithi047 ceph-mon[35327]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:01.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:01 smithi038 ceph-mon[31227]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:01.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:01 smithi179 ceph-mon[35781]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:01.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:01 smithi047 ceph-mon[35327]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:03.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:03 smithi047 ceph-mon[35327]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:37:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:03 smithi038 ceph-mon[31227]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:37:03.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:03 smithi179 ceph-mon[35781]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-19T05:37:05.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:05 smithi038 ceph-mon[31227]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:05 smithi179 ceph-mon[35781]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:05 smithi047 ceph-mon[35327]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:06.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:37:06.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:06.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:37:06.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:06.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:37:06.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:07.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:07 smithi038 ceph-mon[31227]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:37:07.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:07 smithi038 ceph-mon[31227]: Deploying daemon osd.9 on smithi038 2024-09-19T05:37:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:07 smithi179 ceph-mon[35781]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:37:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:07 smithi179 ceph-mon[35781]: Deploying daemon osd.9 on smithi038 2024-09-19T05:37:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:07 smithi047 ceph-mon[35327]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T05:37:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:07 smithi047 ceph-mon[35327]: Deploying daemon osd.9 on smithi038 2024-09-19T05:37:09.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:09 smithi038 ceph-mon[31227]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T05:37:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:09 smithi179 ceph-mon[35781]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T05:37:09.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:09 smithi047 ceph-mon[35327]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T05:37:11.348 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:11 smithi038 ceph-mon[31227]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:11 smithi179 ceph-mon[35781]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:11.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:11 smithi047 ceph-mon[35327]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:12.304 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:12.304 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:12.304 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:12.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:12.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:12.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:13 smithi038 ceph-mon[31227]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:13.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:13 smithi179 ceph-mon[35781]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:13.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:13 smithi047 ceph-mon[35327]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:15.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.413+0000 7f43197fa640 1 -- 172.21.15.38:0/2892981056 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f431c062630 con 0x7f43040a3270 2024-09-19T05:37:15.416 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 9 on host 'smithi038' 2024-09-19T05:37:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.415+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 msgr2=0x7f43040a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.415+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 0x7f43040a5730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f430c005fd0 tx=0x7f430c00b390 comp rx=0 tx=0).stop 2024-09-19T05:37:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 msgr2=0x7f431c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4314005e30 tx=0x7f4314031ee0 comp rx=0 tx=0).stop 2024-09-19T05:37:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 shutdown_connections 2024-09-19T05:37:15.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f43040a3270 0x7f43040a5730 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:15.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f431c1037c0 0x7f431c19f4e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:15.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c102e00 0x7f431c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:15.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 --2- 172.21.15.38:0/2892981056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f431c101c00 0x7f431c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:15.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 >> 172.21.15.38:0/2892981056 conn(0x7f431c0fd390 msgr2=0x7f431c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:37:15.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 shutdown_connections 2024-09-19T05:37:15.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:37:15.416+0000 7f43229ab640 1 -- 172.21.15.38:0/2892981056 wait complete. 2024-09-19T05:37:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:15.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:15.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:15.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.107 DEBUG:teuthology.orchestra.run.smithi038:osd.9> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T05:37:16.110 INFO:tasks.cephadm:Deploying osd.10 on smithi047 with /dev/vg_nvme/lv_1... 2024-09-19T05:37:16.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:37:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:37:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:16.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:16.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:17.562 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:17 smithi038 ceph-mon[31227]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:17.563 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:37:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:37:17.460+0000 7f6357790640 -1 osd.9 0 log_to_monitors true 2024-09-19T05:37:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:17 smithi179 ceph-mon[35781]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:17 smithi047 ceph-mon[35327]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:18.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:18 smithi179 ceph-mon[35781]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:37:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:18 smithi047 ceph-mon[35327]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:37:18.769 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:18 smithi038 ceph-mon[31227]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:37:19.038 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-19T05:37:19.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-09-19T05:37:19.317 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:19.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.449 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:37:19.433+0000 7f6353713640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T05:37:19.449 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:37:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:37:19.445+0000 7f634f53b640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.672 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:19.673 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:19 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:37:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:37:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-19T05:37:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:19 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:19.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 -- 172.21.15.47:0/99595853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc0761a0 msgr2=0x7fa2cc074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:19.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 --2- 172.21.15.47:0/99595853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc074600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009900 tx=0x7fa2bc02a410 comp rx=0 tx=0).stop 2024-09-19T05:37:19.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 -- 172.21.15.47:0/99595853 shutdown_connections 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 --2- 172.21.15.47:0/99595853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc109a60 0x7fa2cc10bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 --2- 172.21.15.47:0/99595853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc074b40 0x7fa2cc074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 --2- 172.21.15.47:0/99595853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc074600 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 -- 172.21.15.47:0/99595853 >> 172.21.15.47:0/99595853 conn(0x7fa2cc0fd3c0 msgr2=0x7fa2cc0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 -- 172.21.15.47:0/99595853 shutdown_connections 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.755+0000 7fa2d1452640 1 -- 172.21.15.47:0/99595853 wait complete. 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 Processor -- start 2024-09-19T05:37:19.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 -- start start 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc074b40 0x7fa2cc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10fb40 con 0x7fa2cc0761a0 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10f9c0 con 0x7fa2cc109a60 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2d1452640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc10fcc0 con 0x7fa2cc074b40 2024-09-19T05:37:19.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38980/0 (socket says 172.21.15.47:38980) 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 -- 172.21.15.47:0/1803346913 learned_addr learned my addr 172.21.15.47:0/1803346913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2caffd640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc074b40 0x7fa2cc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2c27fc640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 -- 172.21.15.47:0/1803346913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc074b40 msgr2=0x7fa2cc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc074b40 0x7fa2cc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 -- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 msgr2=0x7fa2cc19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2cb7fe640 1 -- 172.21.15.47:0/1803346913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc1a1e10 con 0x7fa2cc109a60 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.756+0000 7fa2c27fc640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2cb7fe640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400b970 tx=0x7fa2c400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4019070 con 0x7fa2cc109a60 2024-09-19T05:37:19.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c400cb40 con 0x7fa2cc109a60 2024-09-19T05:37:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc1a2100 con 0x7fa2cc109a60 2024-09-19T05:37:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4007650 con 0x7fa2cc109a60 2024-09-19T05:37:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.757+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1a26c0 con 0x7fa2cc109a60 2024-09-19T05:37:19.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.758+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc108870 con 0x7fa2cc109a60 2024-09-19T05:37:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.759+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa2c401c070 con 0x7fa2cc109a60 2024-09-19T05:37:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.760+0000 7fa2c8ff9640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 0x7fa2b80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.760+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7fa2c40ceaa0 con 0x7fa2cc109a60 2024-09-19T05:37:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.760+0000 7fa2caffd640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 0x7fa2b80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.760+0000 7fa2caffd640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 0x7fa2b80a5730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:37:19.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.763+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2c409c0e0 con 0x7fa2cc109a60 2024-09-19T05:37:19.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:19.947+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa2cc062630 con 0x7fa2b80a3270 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi038 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.617 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.618 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:20.618 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:20.618 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:20 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:20.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi038 2024-09-19T05:37:20.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T05:37:20.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:20.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:20 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi038 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:20.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:20 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:37:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:37:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] boot 2024-09-19T05:37:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:37:21.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] boot 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:37:21.704 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: osd.9 [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] boot 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T05:37:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/4260508949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]': finished 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T05:37:22.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:22 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/4260508949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]': finished 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T05:37:22.815 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:22 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/4260508949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]: dispatch 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a1ddce0-f377-454c-a5f2-b8ecad532909"}]': finished 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T05:37:22.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:22 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:23 smithi047 ceph-mon[35327]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:23 smithi047 ceph-mon[35327]: from='client.? 172.21.15.47:0/2544884931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:23 smithi038 ceph-mon[31227]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:23 smithi038 ceph-mon[31227]: from='client.? 172.21.15.47:0/2544884931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:23 smithi179 ceph-mon[35781]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:23 smithi179 ceph-mon[35781]: from='client.? 172.21.15.47:0/2544884931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:25 smithi047 ceph-mon[35327]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:25.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:25 smithi038 ceph-mon[31227]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:25 smithi179 ceph-mon[35781]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:27.613 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:27 smithi047 ceph-mon[35327]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:27 smithi038 ceph-mon[31227]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:27 smithi179 ceph-mon[35781]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:29.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:29 smithi047 ceph-mon[35327]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:29 smithi038 ceph-mon[31227]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:29 smithi179 ceph-mon[35781]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:31.608 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:31 smithi047 ceph-mon[35327]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:31.608 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:37:31.608 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:31 smithi038 ceph-mon[31227]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:37:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:31 smithi179 ceph-mon[35781]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:37:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:32.576 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:32 smithi047 ceph-mon[35327]: Deploying daemon osd.10 on smithi047 2024-09-19T05:37:32.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:32 smithi179 ceph-mon[35781]: Deploying daemon osd.10 on smithi047 2024-09-19T05:37:32.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:32 smithi038 ceph-mon[31227]: Deploying daemon osd.10 on smithi047 2024-09-19T05:37:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:33 smithi047 ceph-mon[35327]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:33 smithi038 ceph-mon[31227]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:33 smithi179 ceph-mon[35781]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:35.670 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:35 smithi047 ceph-mon[35327]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:35 smithi038 ceph-mon[31227]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:35 smithi179 ceph-mon[35781]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:37 smithi047 ceph-mon[35327]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:37 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:37 smithi038 ceph-mon[31227]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:37 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:37 smithi179 ceph-mon[35781]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:37 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:39.585 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:39 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.882 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 10 on host 'smithi047' 2024-09-19T05:37:39.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.879+0000 7fa2c8ff9640 1 -- 172.21.15.47:0/1803346913 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa2cc062630 con 0x7fa2b80a3270 2024-09-19T05:37:39.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.881+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 msgr2=0x7fa2b80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:39.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.881+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 0x7fa2b80a5730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc031400 comp rx=0 tx=0).stop 2024-09-19T05:37:39.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.881+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 msgr2=0x7fa2cc1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.881+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400b970 tx=0x7fa2c400be30 comp rx=0 tx=0).stop 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.881+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 shutdown_connections 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa2b80a3270 0x7fa2b80a5730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2cc109a60 0x7fa2cc1a17a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2cc0761a0 0x7fa2cc19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 --2- 172.21.15.47:0/1803346913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2cc074b40 0x7fa2cc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 >> 172.21.15.47:0/1803346913 conn(0x7fa2cc0fd3c0 msgr2=0x7fa2cc105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 shutdown_connections 2024-09-19T05:37:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-19T05:37:39.882+0000 7fa2d1452640 1 -- 172.21.15.47:0/1803346913 wait complete. 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:39 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:39.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:39.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:39.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:39 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.462 DEBUG:teuthology.orchestra.run.smithi047:osd.10> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T05:37:40.465 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-09-19T05:37:40.466 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:37:40.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:40.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.729 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:37:40.850 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:40.850 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.850 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:37:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:40.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:41.589 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:41 smithi047 ceph-mon[35327]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:41.871 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:41 smithi179 ceph-mon[35781]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:41 smithi038 ceph-mon[31227]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:42.971 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:37:42 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:37:42.653+0000 7fc0b34f9640 -1 osd.10 0 log_to_monitors true 2024-09-19T05:37:43.457 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-09-19T05:37:43.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.717 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:43.718 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:43.718 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:43 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:43.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:43 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.748 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:37:43.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:43.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:37:43.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:43 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:37:44.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.282+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/521820389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64100ad0 msgr2=0x7f1a64100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:44.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.282+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/521820389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64100ad0 0x7f1a64100ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5402a410 comp rx=0 tx=0).stop 2024-09-19T05:37:44.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/521820389 shutdown_connections 2024-09-19T05:37:44.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/521820389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64102690 0x7f1a64108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/521820389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 0x7f1a64102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/521820389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64100ad0 0x7f1a64100ed0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/521820389 >> 172.21.15.179:0/521820389 conn(0x7f1a640fc280 msgr2=0x7f1a640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/521820389 shutdown_connections 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/521820389 wait complete. 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 Processor -- start 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 -- start start 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.283+0000 7f1a6c5fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64100ad0 0x7f1a64104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 0x7f1a641030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410c760 con 0x7f1a64101cd0 2024-09-19T05:37:44.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410c5e0 con 0x7f1a64100ad0 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6410c8e0 con 0x7f1a64102690 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38312/0 (socket says 172.21.15.179:38312) 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 -- 172.21.15.179:0/2915927766 learned_addr learned my addr 172.21.15.179:0/2915927766 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a69b98640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 0x7f1a641030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6a399640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64100ad0 0x7f1a64104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 -- 172.21.15.179:0/2915927766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64100ad0 msgr2=0x7f1a64104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64100ad0 0x7f1a64104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 -- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 msgr2=0x7f1a641030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 0x7f1a641030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 -- 172.21.15.179:0/2915927766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a64103eb0 con 0x7f1a64102690 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6ab9a640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00cad0 tx=0x7f1a5c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:37:44.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00ae20 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a641a20d0 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.284+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5c005020 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.285+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00f6b0 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.285+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641a2630 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.286+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a641a28c0 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.287+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1a5c0051d0 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.287+0000 7f1a5b7fe640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 0x7f1a500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.287+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6398+0+0 (secure 0 0 0) 0x7f1a5c0cf410 con 0x7f1a64102690 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.288+0000 7f1a6a399640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 0x7f1a500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:37:44.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.288+0000 7f1a6a399640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 0x7f1a500a5700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005ab0 tx=0x7f1a54000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:37:44.297 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.296+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a5c014030 con 0x7f1a64102690 2024-09-19T05:37:44.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:37:44.477+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1a641a2a90 con 0x7f1a500a3240 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi047 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi047 2024-09-19T05:37:44.895 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:37:44.896 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:44.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:37:44.526+0000 7fc0af47c640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T05:37:44.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:37:44.548+0000 7fc0ab2a4640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:37:44.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi047 2024-09-19T05:37:44.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-19T05:37:44.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:37:44.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:37:44.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:37:44.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:45 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:45 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.935 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:45 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.935 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:45 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:45 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:45.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:45 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] boot 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:46.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:46 smithi038 ceph-mon[31227]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:46.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:37:46.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:37:46.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] boot 2024-09-19T05:37:46.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T05:37:46.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:46.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:46 smithi179 ceph-mon[35781]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: osd.10 [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] boot 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:37:46.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:46 smithi047 ceph-mon[35327]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/971493338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]': finished 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:47 smithi038 ceph-mon[31227]: from='client.? 172.21.15.179:0/2240998372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:37:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/971493338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]': finished 2024-09-19T05:37:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:37:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:47 smithi179 ceph-mon[35781]: from='client.? 172.21.15.179:0/2240998372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:37:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/971493338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]: dispatch 2024-09-19T05:37:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7cda5e25-4843-426b-a917-89f6546f30a8"}]': finished 2024-09-19T05:37:47.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T05:37:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:47 smithi047 ceph-mon[35327]: from='client.? 172.21.15.179:0/2240998372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:37:48.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:48 smithi038 ceph-mon[31227]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:48.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:48 smithi179 ceph-mon[35781]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:48.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:48 smithi047 ceph-mon[35327]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:49 smithi038 ceph-mon[31227]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T05:37:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:49 smithi179 ceph-mon[35781]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T05:37:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:49.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:49 smithi047 ceph-mon[35327]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T05:37:49.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:50.877 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:50 smithi047 ceph-mon[35327]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:50.877 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:50 smithi047 ceph-mon[35327]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:37:50.877 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:50.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:50 smithi038 ceph-mon[31227]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:50.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:50 smithi038 ceph-mon[31227]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:37:50.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:50 smithi179 ceph-mon[35781]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:37:50.985 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:50 smithi179 ceph-mon[35781]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:37:50.985 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:37:52.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:52 smithi179 ceph-mon[35781]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T05:37:52.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:52 smithi038 ceph-mon[31227]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T05:37:52.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:52 smithi047 ceph-mon[35327]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T05:37:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:54 smithi038 ceph-mon[31227]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:37:54.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:54 smithi179 ceph-mon[35781]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:37:54.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:54 smithi047 ceph-mon[35327]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-19T05:37:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:55 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:37:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:55 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:55 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:37:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:55 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:55 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:37:55.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:55 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:37:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:56 smithi179 ceph-mon[35781]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:56 smithi179 ceph-mon[35781]: Deploying daemon osd.11 on smithi179 2024-09-19T05:37:56.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:56 smithi047 ceph-mon[35327]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:56 smithi047 ceph-mon[35327]: Deploying daemon osd.11 on smithi179 2024-09-19T05:37:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:56 smithi038 ceph-mon[31227]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:37:57.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:56 smithi038 ceph-mon[31227]: Deploying daemon osd.11 on smithi179 2024-09-19T05:37:58.830 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:37:58 smithi179 ceph-mon[35781]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:37:58.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:37:58 smithi047 ceph-mon[35327]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:37:59.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:37:58 smithi038 ceph-mon[31227]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T05:38:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:00 smithi179 ceph-mon[35781]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:38:00.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:00 smithi047 ceph-mon[35327]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:38:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:00 smithi038 ceph-mon[31227]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T05:38:02.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:02 smithi179 ceph-mon[35781]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:38:02.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:02.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:02.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:02.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:02 smithi047 ceph-mon[35327]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:38:02.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:02.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:02.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:03.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:02 smithi038 ceph-mon[31227]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:38:03.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:03.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:03.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:04.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:04 smithi179 ceph-mon[35781]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:04.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:04 smithi047 ceph-mon[35327]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:04 smithi038 ceph-mon[31227]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:05.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.665+0000 7f1a5b7fe640 1 -- 172.21.15.179:0/2915927766 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1a641a2a90 con 0x7f1a500a3240 2024-09-19T05:38:05.668 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-09-19T05:38:05.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 msgr2=0x7f1a500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:05.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 0x7f1a500a5700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005ab0 tx=0x7f1a54000bc0 comp rx=0 tx=0).stop 2024-09-19T05:38:05.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 msgr2=0x7f1a641035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:05.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00cad0 tx=0x7f1a5c00cf90 comp rx=0 tx=0).stop 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 shutdown_connections 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1a500a3240 0x7f1a500a5700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a64102690 0x7f1a641035f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a64101cd0 0x7f1a641030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 --2- 172.21.15.179:0/2915927766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a64100ad0 0x7f1a64104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 >> 172.21.15.179:0/2915927766 conn(0x7f1a640fc280 msgr2=0x7f1a6406a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 shutdown_connections 2024-09-19T05:38:05.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:05.667+0000 7f1a6c5fd640 1 -- 172.21.15.179:0/2915927766 wait complete. 2024-09-19T05:38:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:06.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:06.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.230 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:06.230 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.230 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:06.369 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T05:38:06.372 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T05:38:06.372 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:38:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:06 smithi179 ceph-mon[35781]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:07.024 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:06 smithi038 ceph-mon[31227]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:07.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 -- 172.21.15.38:0/2382211276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff304102ef0 msgr2=0x7ff304103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:07.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 --2- 172.21.15.38:0/2382211276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff304102ef0 0x7ff304103370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009930 tx=0x7ff2f802a420 comp rx=0 tx=0).stop 2024-09-19T05:38:07.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 -- 172.21.15.38:0/2382211276 shutdown_connections 2024-09-19T05:38:07.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 --2- 172.21.15.38:0/2382211276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3041038b0 0x7ff30410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 --2- 172.21.15.38:0/2382211276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff304102ef0 0x7ff304103370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 --2- 172.21.15.38:0/2382211276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3040fff10 0x7ff304100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 -- 172.21.15.38:0/2382211276 >> 172.21.15.38:0/2382211276 conn(0x7ff3040fba80 msgr2=0x7ff3040fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 -- 172.21.15.38:0/2382211276 shutdown_connections 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.107+0000 7ff309e53640 1 -- 172.21.15.38:0/2382211276 wait complete. 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 Processor -- start 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 -- start start 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff304102ef0 0x7ff30419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 0x7ff3041a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410d880 con 0x7ff3040fff10 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410d700 con 0x7ff304102ef0 2024-09-19T05:38:07.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff309e53640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410da00 con 0x7ff3041038b0 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47340/0 (socket says 172.21.15.38:47340) 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 -- 172.21.15.38:0/1013194078 learned_addr learned my addr 172.21.15.38:0/1013194078 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff303fff640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 0x7ff3041a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff302ffd640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff304102ef0 0x7ff30419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 -- 172.21.15.38:0/1013194078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 msgr2=0x7ff3041a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 0x7ff3041a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 -- 172.21.15.38:0/1013194078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff304102ef0 msgr2=0x7ff30419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.108+0000 7ff3037fe640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff304102ef0 0x7ff30419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff3037fe640 1 -- 172.21.15.38:0/1013194078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3041a1d50 con 0x7ff3040fff10 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff303fff640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 0x7ff3041a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff3037fe640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d950 tx=0x7ff2f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4017070 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4009c30 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3041a1fe0 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f600 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.109+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a2440 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.110+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3041a2730 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.112+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff2f401a070 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.113+0000 7ff300ff9640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 0x7ff2ec0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.113+0000 7ff302ffd640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 0x7ff2ec0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.113+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7ff2f40cee20 con 0x7ff3040fff10 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.114+0000 7ff302ffd640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 0x7ff2ec0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:07.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.116+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2f409c0c0 con 0x7ff3040fff10 2024-09-19T05:38:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:06 smithi047 ceph-mon[35327]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:07.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.295+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff3041a2900 con 0x7ff3040fff10 2024-09-19T05:38:07.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.296+0000 7ff300ff9640 1 -- 172.21.15.38:0/1013194078 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7ff2f409b9c0 con 0x7ff3040fff10 2024-09-19T05:38:07.297 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:07.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.298+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 msgr2=0x7ff2ec0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:07.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.298+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 0x7ff2ec0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f8000bc0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.298+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 msgr2=0x7ff30419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.298+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400d950 tx=0x7ff2f400de10 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.298+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 shutdown_connections 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff2ec0a3270 0x7ff2ec0a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3041038b0 0x7ff3041a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff304102ef0 0x7ff30419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 --2- 172.21.15.38:0/1013194078 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3040fff10 0x7ff30419a120 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 >> 172.21.15.38:0/1013194078 conn(0x7ff3040fba80 msgr2=0x7ff3041086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 shutdown_connections 2024-09-19T05:38:07.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:07.299+0000 7ff309e53640 1 -- 172.21.15.38:0/1013194078 wait complete. 2024-09-19T05:38:08.080 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:07 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1013194078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:08.108 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726724265,"num_in_osds":12,"osd_in_since":1726724266,"num_remapped_pgs":0} 2024-09-19T05:38:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:07 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1013194078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:08.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:38:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:38:08.004+0000 7f3890619640 -1 osd.11 0 log_to_monitors true 2024-09-19T05:38:08.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:07 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1013194078' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:09.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:38:09.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:08 smithi038 ceph-mon[31227]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:09.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:08 smithi038 ceph-mon[31227]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:08 smithi038 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:08 smithi179 ceph-mon[35781]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:08 smithi179 ceph-mon[35781]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:08 smithi179 ceph-mon[35781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:08 smithi047 ceph-mon[35327]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:08 smithi047 ceph-mon[35327]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:08 smithi047 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:38:09.398 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 -- 172.21.15.38:0/3204534204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec101c00 msgr2=0x7fa4ec102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 --2- 172.21.15.38:0/3204534204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec101c00 0x7fa4ec102000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009900 tx=0x7fa4e802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 -- 172.21.15.38:0/3204534204 shutdown_connections 2024-09-19T05:38:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 --2- 172.21.15.38:0/3204534204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 --2- 172.21.15.38:0/3204534204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 0x7fa4ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 --2- 172.21.15.38:0/3204534204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec101c00 0x7fa4ec102000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 -- 172.21.15.38:0/3204534204 >> 172.21.15.38:0/3204534204 conn(0x7fa4ec0fd390 msgr2=0x7fa4ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 -- 172.21.15.38:0/3204534204 shutdown_connections 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.883+0000 7fa4f3926640 1 -- 172.21.15.38:0/3204534204 wait complete. 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 Processor -- start 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 -- start start 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 0x7fa4ec197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 0x7fa4ec198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec10d8e0 con 0x7fa4ec1037c0 2024-09-19T05:38:09.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec10d760 con 0x7fa4ec101c00 2024-09-19T05:38:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f3926640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec10da60 con 0x7fa4ec102e00 2024-09-19T05:38:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f1ec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f1ec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54564/0 (socket says 172.21.15.38:54564) 2024-09-19T05:38:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f1ec3640 1 -- 172.21.15.38:0/1403556146 learned_addr learned my addr 172.21.15.38:0/1403556146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f0ec1640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 0x7fa4ec198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.884+0000 7fa4f16c2640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 0x7fa4ec197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 -- 172.21.15.38:0/1403556146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 msgr2=0x7fa4ec198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 0x7fa4ec198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 -- 172.21.15.38:0/1403556146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 msgr2=0x7fa4ec197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 0x7fa4ec197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 -- 172.21.15.38:0/1403556146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec19fb10 con 0x7fa4ec1037c0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f16c2640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 0x7fa4ec197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f1ec3640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b970 tx=0x7fa4e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4019070 con 0x7fa4ec1037c0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4014440 con 0x7fa4ec1037c0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec19fe00 con 0x7fa4ec1037c0 2024-09-19T05:38:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40075d0 con 0x7fa4ec1037c0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.885+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a02e0 con 0x7fa4ec1037c0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.886+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec109000 con 0x7fa4ec1037c0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.887+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa4e401c070 con 0x7fa4ec1037c0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.888+0000 7fa4e27fc640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 0x7fa4d80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.888+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7fa4e40cede0 con 0x7fa4ec1037c0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.888+0000 7fa4f16c2640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 0x7fa4d80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.889+0000 7fa4f16c2640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 0x7fa4d80a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006010 tx=0x7fa4e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:09.891+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4e409c020 con 0x7fa4ec1037c0 2024-09-19T05:38:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.066+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa4ec02da10 con 0x7fa4ec1037c0 2024-09-19T05:38:10.068 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:38:10.068 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: Detected new or changed devices on smithi179 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi179 to 4272M 2024-09-19T05:38:10.069 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:10.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:10.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.066+0000 7fa4e27fc640 1 -- 172.21.15.38:0/1403556146 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fa4e409b920 con 0x7fa4ec1037c0 2024-09-19T05:38:10.070 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 msgr2=0x7fa4d80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 0x7fa4d80a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006010 tx=0x7fa4e8000bc0 comp rx=0 tx=0).stop 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 msgr2=0x7fa4ec19f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b970 tx=0x7fa4e400bd70 comp rx=0 tx=0).stop 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 shutdown_connections 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4d80a3270 0x7fa4d80a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:10.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4ec1037c0 0x7fa4ec19f4a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4ec102e00 0x7fa4ec198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 --2- 172.21.15.38:0/1403556146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4ec101c00 0x7fa4ec197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.068+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 >> 172.21.15.38:0/1403556146 conn(0x7fa4ec0fd390 msgr2=0x7fa4ec1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.069+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 shutdown_connections 2024-09-19T05:38:10.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:10.069+0000 7fa4f3926640 1 -- 172.21.15.38:0/1403556146 wait complete. 2024-09-19T05:38:10.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:38:09.854+0000 7f388c59c640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T05:38:10.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:38:09.865+0000 7f38883c4640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: Detected new or changed devices on smithi179 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi179 to 4272M 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:10.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-09-19T05:38:10.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: Detected new or changed devices on smithi179 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi179 to 4272M 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:10.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:10.734 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726724265,"num_in_osds":12,"osd_in_since":1726724266,"num_remapped_pgs":0} 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1403556146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:38:11.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:38:11.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1403556146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1403556146' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:11.736 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:38:12.007 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: purged_snaps scrub starts 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: purged_snaps scrub ok 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] boot 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:38:12.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: purged_snaps scrub starts 2024-09-19T05:38:12.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: purged_snaps scrub ok 2024-09-19T05:38:12.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] boot 2024-09-19T05:38:12.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:38:12.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: purged_snaps scrub starts 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: purged_snaps scrub ok 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: osd.11 [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] boot 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T05:38:12.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 -- 172.21.15.38:0/105693022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 msgr2=0x7f2690100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 --2- 172.21.15.38:0/105693022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690100ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f268c009900 tx=0x7f268c02a410 comp rx=0 tx=0).stop 2024-09-19T05:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 -- 172.21.15.38:0/105693022 shutdown_connections 2024-09-19T05:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 --2- 172.21.15.38:0/105693022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690102690 0x7f2690108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 --2- 172.21.15.38:0/105693022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690101cd0 0x7f2690102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 --2- 172.21.15.38:0/105693022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690100ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 -- 172.21.15.38:0/105693022 >> 172.21.15.38:0/105693022 conn(0x7f26900fc280 msgr2=0x7f26900fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 -- 172.21.15.38:0/105693022 shutdown_connections 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 -- 172.21.15.38:0/105693022 wait complete. 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.467+0000 7f2698509640 1 Processor -- start 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 -- start start 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 0x7f26901032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 0x7f26901037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f269010c730 con 0x7f2690100ad0 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f269010c5b0 con 0x7f2690102690 2024-09-19T05:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2698509640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f269010c8b0 con 0x7f2690101cd0 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54588/0 (socket says 172.21.15.38:54588) 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 -- 172.21.15.38:0/2465483458 learned_addr learned my addr 172.21.15.38:0/2465483458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2695aa4640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 0x7f26901032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2696aa6640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 0x7f26901037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 -- 172.21.15.38:0/2465483458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 msgr2=0x7f26901032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 0x7f26901032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 -- 172.21.15.38:0/2465483458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 msgr2=0x7f26901037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 0x7f26901037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 -- 172.21.15.38:0/2465483458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901040a0 con 0x7f2690100ad0 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2695aa4640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 0x7f26901032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f2696aa6640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 0x7f26901037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:12.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.468+0000 7f26962a5640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f268c031610 tx=0x7f268c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.469+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c03f030 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.469+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268c003ce0 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.469+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901a2030 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.469+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c035070 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.469+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a24e0 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.470+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26901a2710 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.471+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f268c042030 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.472+0000 7f26877fe640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 0x7f267c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.472+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f268c0efd50 con 0x7f2690100ad0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.472+0000 7f2695aa4640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 0x7f267c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.472+0000 7f2695aa4640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 0x7f267c0a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2680005d00 tx=0x7f2680005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:12.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.475+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f268c0bcf10 con 0x7f2690100ad0 2024-09-19T05:38:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.652+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f26901a28e0 con 0x7f2690100ad0 2024-09-19T05:38:12.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.652+0000 7f26877fe640 1 -- 172.21.15.38:0/2465483458 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f268c0bc810 con 0x7f2690100ad0 2024-09-19T05:38:12.654 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 msgr2=0x7f267c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 0x7f267c0a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2680005d00 tx=0x7f2680005c70 comp rx=0 tx=0).stop 2024-09-19T05:38:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 msgr2=0x7f2690104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:12.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f268c031610 tx=0x7f268c0043b0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 shutdown_connections 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f267c0a3240 0x7f267c0a5700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2690102690 0x7f26901037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2690101cd0 0x7f26901032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 --2- 172.21.15.38:0/2465483458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2690100ad0 0x7f2690104c50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.654+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 >> 172.21.15.38:0/2465483458 conn(0x7f26900fc280 msgr2=0x7f2690069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.655+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 shutdown_connections 2024-09-19T05:38:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:12.655+0000 7f2698509640 1 -- 172.21.15.38:0/2465483458 wait complete. 2024-09-19T05:38:13.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:12 smithi038 ceph-mon[31227]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:13.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:12 smithi038 ceph-mon[31227]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:38:13.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:12 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2465483458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:13.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:12 smithi179 ceph-mon[35781]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:13.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:12 smithi179 ceph-mon[35781]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:38:13.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:12 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2465483458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:13.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:12 smithi047 ceph-mon[35327]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:38:13.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:12 smithi047 ceph-mon[35327]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:38:13.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:12 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2465483458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:38:13.380 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726724290,"num_in_osds":12,"osd_in_since":1726724266,"num_remapped_pgs":0} 2024-09-19T05:38:13.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:38:13.637 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:14.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 -- 172.21.15.38:0/1856802397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c101de0 msgr2=0x7f3f5c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1856802397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c101de0 0x7f3f5c1021c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009900 tx=0x7f3f5802a410 comp rx=0 tx=0).stop 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 -- 172.21.15.38:0/1856802397 shutdown_connections 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1856802397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c106f10 0x7f3f5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1856802397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c102790 0x7f3f5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1856802397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c101de0 0x7f3f5c1021c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 -- 172.21.15.38:0/1856802397 >> 172.21.15.38:0/1856802397 conn(0x7f3f5c0fd8c0 msgr2=0x7f3f5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 -- 172.21.15.38:0/1856802397 shutdown_connections 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.133+0000 7f3f646e3640 1 -- 172.21.15.38:0/1856802397 wait complete. 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 Processor -- start 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 -- start start 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 0x7f3f5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c106f10 0x7f3f5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c111f70 con 0x7f3f5c102790 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c111df0 con 0x7f3f5c101de0 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f646e3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c1120f0 con 0x7f3f5c106f10 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f61c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f61c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54608/0 (socket says 172.21.15.38:54608) 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f61c7e640 1 -- 172.21.15.38:0/1941460661 learned_addr learned my addr 172.21.15.38:0/1941460661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f6247f640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 0x7f3f5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.134+0000 7f3f62c80640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c106f10 0x7f3f5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 -- 172.21.15.38:0/1941460661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c106f10 msgr2=0x7f3f5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c106f10 0x7f3f5c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 -- 172.21.15.38:0/1941460661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 msgr2=0x7f3f5c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 0x7f3f5c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 -- 172.21.15.38:0/1941460661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c1a68f0 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f6247f640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 0x7f3f5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f61c7e640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000d770 tx=0x7f3f5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50017070 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c1a6be0 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f50005500 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50004e00 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.135+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1a70d0 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.136+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c1021c0 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.137+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3f50004fa0 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.138+0000 7f3f4f7fe640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 0x7f3f480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.138+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3f500cf280 con 0x7f3f5c102790 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.138+0000 7f3f6247f640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 0x7f3f480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.139+0000 7f3f6247f640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 0x7f3f480a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005ab0 tx=0x7f3f58005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:14.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.141+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f5009c440 con 0x7f3f5c102790 2024-09-19T05:38:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.321+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f5c075a20 con 0x7f3f5c102790 2024-09-19T05:38:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.322+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/1941460661 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18446 (secure 0 0 0) 0x7f3f5c075a20 con 0x7f3f5c102790 2024-09-19T05:38:14.324 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:14.324 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":76,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:38:11.857303+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 msgr2=0x7f3f480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 0x7f3f480a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005ab0 tx=0x7f3f58005a20 comp rx=0 tx=0).stop 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 msgr2=0x7f3f5c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000d770 tx=0x7f3f5000dc30 comp rx=0 tx=0).stop 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 shutdown_connections 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3f480a3240 0x7f3f480a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f5c106f10 0x7f3f5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f5c102790 0x7f3f5c074a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 --2- 172.21.15.38:0/1941460661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c101de0 0x7f3f5c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 >> 172.21.15.38:0/1941460661 conn(0x7f3f5c0fd8c0 msgr2=0x7f3f5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 shutdown_connections 2024-09-19T05:38:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:14.324+0000 7f3f646e3640 1 -- 172.21.15.38:0/1941460661 wait complete. 2024-09-19T05:38:15.059 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:14 smithi038 ceph-mon[31227]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:15.059 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:14 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1941460661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:38:15.094 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:34:23.998201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:38:15.095 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T05:38:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:14 smithi179 ceph-mon[35781]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:14 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1941460661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:38:15.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:14 smithi047 ceph-mon[35327]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:15.228 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:14 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1941460661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:38:15.352 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 -- 172.21.15.38:0/122412604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041082a0 msgr2=0x7fed0410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 --2- 172.21.15.38:0/122412604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041082a0 0x7fed0410ec40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009cd0 tx=0x7fecfc02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 -- 172.21.15.38:0/122412604 shutdown_connections 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 --2- 172.21.15.38:0/122412604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041082a0 0x7fed0410ec40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 --2- 172.21.15.38:0/122412604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041036e0 0x7fed04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 --2- 172.21.15.38:0/122412604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 0x7fed04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 -- 172.21.15.38:0/122412604 >> 172.21.15.38:0/122412604 conn(0x7fed040fc7d0 msgr2=0x7fed040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:15.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 -- 172.21.15.38:0/122412604 shutdown_connections 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.807+0000 7fed0c556640 1 -- 172.21.15.38:0/122412604 wait complete. 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 Processor -- start 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 -- start start 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 0x7fed04100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041082a0 0x7fed04101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed041120a0 con 0x7fed041036e0 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fed04111f20 con 0x7fed04102d30 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0c556640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed04112220 con 0x7fed041082a0 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed09af1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed09af1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54624/0 (socket says 172.21.15.38:54624) 2024-09-19T05:38:15.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed09af1640 1 -- 172.21.15.38:0/2809180063 learned_addr learned my addr 172.21.15.38:0/2809180063 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.808+0000 7fed0a2f2640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 0x7fed04100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed0aaf3640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041082a0 0x7fed04101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 -- 172.21.15.38:0/2809180063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041082a0 msgr2=0x7fed04101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041082a0 0x7fed04101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 -- 172.21.15.38:0/2809180063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 msgr2=0x7fed04100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 0x7fed04100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 -- 172.21.15.38:0/2809180063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041a6a10 con 0x7fed041036e0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed09af1640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d750 tx=0x7fecf400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4017070 con 0x7fed041036e0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4005500 con 0x7fed041036e0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041a6d00 con 0x7fed041036e0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004e00 con 0x7fed041036e0 2024-09-19T05:38:15.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.809+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a7190 con 0x7fed041036e0 2024-09-19T05:38:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.810+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04103110 con 0x7fed041036e0 2024-09-19T05:38:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.811+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fecf4004fa0 con 0x7fed041036e0 2024-09-19T05:38:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.812+0000 7fecfb7fe640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 0x7fecf00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.812+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fecf40cf1f0 con 0x7fed041036e0 2024-09-19T05:38:15.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.812+0000 7fed0a2f2640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 0x7fecf00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:15.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.813+0000 7fed0a2f2640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 0x7fecf00a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed0000bbe0 tx=0x7fed0000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:15.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.815+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecf409c310 con 0x7fed041036e0 2024-09-19T05:38:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.990+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed0410a2b0 con 0x7fed041036e0 2024-09-19T05:38:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.991+0000 7fecfb7fe640 1 -- 172.21.15.38:0/2809180063 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fecf409bc10 con 0x7fed041036e0 2024-09-19T05:38:15.993 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 msgr2=0x7fecf00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 0x7fecf00a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed0000bbe0 tx=0x7fed0000b390 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 msgr2=0x7fed04100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d750 tx=0x7fecf400dc10 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 shutdown_connections 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fecf00a3240 0x7fecf00a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed041082a0 0x7fed04101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed041036e0 0x7fed04100ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 --2- 172.21.15.38:0/2809180063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed04102d30 0x7fed04100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 >> 172.21.15.38:0/2809180063 conn(0x7fed040fc7d0 msgr2=0x7fed04106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 shutdown_connections 2024-09-19T05:38:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:15.993+0000 7fed0c556640 1 -- 172.21.15.38:0/2809180063 wait complete. 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.a on smithi038 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.d on smithi038 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.g on smithi038 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.j on smithi038 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.b on smithi047 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.e on smithi047 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.h on smithi047 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.k on smithi047 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-09-19T05:38:16.672 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-09-19T05:38:16.673 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-09-19T05:38:16.673 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-09-19T05:38:16.929 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:38:17.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:16 smithi179 ceph-mon[35781]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:17.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:16 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2809180063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:38:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:16 smithi038 ceph-mon[31227]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:16 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2809180063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:38:17.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:16 smithi047 ceph-mon[35327]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:17.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:16 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2809180063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:38:17.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 -- 172.21.15.179:0/2396578205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 msgr2=0x7f5698103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:17.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 --2- 172.21.15.179:0/2396578205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f5698103280 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f568c009900 tx=0x7f568c02a410 comp rx=0 tx=0).stop 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 -- 172.21.15.179:0/2396578205 shutdown_connections 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 --2- 172.21.15.179:0/2396578205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f569810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 --2- 172.21.15.179:0/2396578205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f5698103280 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 --2- 172.21.15.179:0/2396578205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 -- 172.21.15.179:0/2396578205 >> 172.21.15.179:0/2396578205 conn(0x7f56980fd390 msgr2=0x7f56980ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 -- 172.21.15.179:0/2396578205 shutdown_connections 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.391+0000 7f569c94f640 1 -- 172.21.15.179:0/2396578205 wait complete. 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 Processor -- start 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 -- start start 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:17.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f5698074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f569810d880 con 0x7f56981037c0 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f569810d700 con 0x7f5698101c00 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569c94f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f569810da00 con 0x7f5698102e00 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:60128/0 (socket says 172.21.15.179:60128) 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 -- 172.21.15.179:0/2670199965 learned_addr learned my addr 172.21.15.179:0/2670199965 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569659c640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5696d9d640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f5698074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 -- 172.21.15.179:0/2670199965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 msgr2=0x7f5698075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 -- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 msgr2=0x7f5698074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f5698074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 -- 172.21.15.179:0/2670199965 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56980753e0 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f569659c640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5696d9d640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f5698074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.392+0000 7f5695d9b640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f568c009230 tx=0x7f568c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.393+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c03f030 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.393+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56981a2040 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.393+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c003ce0 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.393+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c035070 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.393+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981a2540 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.394+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5698102000 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.395+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f568c042030 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.396+0000 7f56877fe640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 0x7f56800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.396+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f568c0efec0 con 0x7f5698102e00 2024-09-19T05:38:17.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.396+0000 7f569659c640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 0x7f56800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:17.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.396+0000 7f569659c640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 0x7f56800a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56880058c0 tx=0x7f5688005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:17.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.400+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f568c0bd080 con 0x7f5698102e00 2024-09-19T05:38:17.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.579+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) -- 0x7f5698062830 con 0x7f56800a3240 2024-09-19T05:38:17.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.587+0000 7f56877fe640 1 -- 172.21.15.179:0/2670199965 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5698062830 con 0x7f56800a3240 2024-09-19T05:38:17.590 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-09-19T05:38:17.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 msgr2=0x7f56800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:17.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 0x7f56800a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56880058c0 tx=0x7f5688005850 comp rx=0 tx=0).stop 2024-09-19T05:38:17.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 msgr2=0x7f56980745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f568c009230 tx=0x7f568c004360 comp rx=0 tx=0).stop 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 shutdown_connections 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f56800a3240 0x7f56800a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56981037c0 0x7f5698074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698102e00 0x7f56980745e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 --2- 172.21.15.179:0/2670199965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5698101c00 0x7f5698075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 >> 172.21.15.179:0/2670199965 conn(0x7f56980fd390 msgr2=0x7f56981061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.589+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 shutdown_connections 2024-09-19T05:38:17.593 INFO:teuthology.orchestra.run.smithi179.stderr:2024-09-19T05:38:17.590+0000 7f569c94f640 1 -- 172.21.15.179:0/2670199965 wait complete. 2024-09-19T05:38:18.300 DEBUG:teuthology.orchestra.run.smithi038:mds.a> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T05:38:18.303 DEBUG:teuthology.orchestra.run.smithi038:mds.d> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T05:38:18.345 DEBUG:teuthology.orchestra.run.smithi038:mds.g> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T05:38:18.389 DEBUG:teuthology.orchestra.run.smithi038:mds.j> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T05:38:18.431 DEBUG:teuthology.orchestra.run.smithi047:mds.b> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T05:38:18.434 DEBUG:teuthology.orchestra.run.smithi047:mds.e> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T05:38:18.478 DEBUG:teuthology.orchestra.run.smithi047:mds.h> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T05:38:18.520 DEBUG:teuthology.orchestra.run.smithi047:mds.k> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T05:38:18.563 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T05:38:18.566 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T05:38:18.609 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T05:38:18.651 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T05:38:18.694 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T05:38:18.695 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:18.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:18 smithi047 ceph-mon[35327]: Deploying daemon mds.l on smithi179 2024-09-19T05:38:18.872 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:18.872 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.873 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:18 smithi179 ceph-mon[35781]: Deploying daemon mds.l on smithi179 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:18.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:18 smithi038 ceph-mon[31227]: Deploying daemon mds.l on smithi179 2024-09-19T05:38:18.958 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 -- 172.21.15.38:0/1195079936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e810c180 msgr2=0x7f44e810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 --2- 172.21.15.38:0/1195079936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e810c180 0x7f44e810e610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009ca0 tx=0x7f44e002a950 comp rx=0 tx=0).stop 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 -- 172.21.15.38:0/1195079936 shutdown_connections 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 --2- 172.21.15.38:0/1195079936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e810c180 0x7f44e810e610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 --2- 172.21.15.38:0/1195079936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e8103700 0x7f44e810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 --2- 172.21.15.38:0/1195079936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8102de0 0x7f44e81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 -- 172.21.15.38:0/1195079936 >> 172.21.15.38:0/1195079936 conn(0x7f44e80fc7a0 msgr2=0x7f44e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 -- 172.21.15.38:0/1195079936 shutdown_connections 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.453+0000 7f44ee50f640 1 -- 172.21.15.38:0/1195079936 wait complete. 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 Processor -- start 2024-09-19T05:38:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 -- start start 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8103700 0x7f44e8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e810c180 0x7f44e81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8111f70 con 0x7f44e8102de0 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8111df0 con 0x7f44e8103700 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.454+0000 7f44ee50f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44e81120f0 con 0x7f44e810c180 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40396/0 (socket says 172.21.15.38:40396) 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 -- 172.21.15.38:0/4131538485 learned_addr learned my addr 172.21.15.38:0/4131538485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e77fe640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8103700 0x7f44e8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44ecaac640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e810c180 0x7f44e81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:19.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 -- 172.21.15.38:0/4131538485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e810c180 msgr2=0x7f44e81014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e810c180 0x7f44e81014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 -- 172.21.15.38:0/4131538485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8103700 msgr2=0x7f44e8100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8103700 0x7f44e8100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 -- 172.21.15.38:0/4131538485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e81a68b0 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e7fff640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d800cd80 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e81a6ba0 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d8010910 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80188f0 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.455+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81a70e0 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.456+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e81031c0 con 0x7f44e8102de0 2024-09-19T05:38:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.458+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f44d8010ac0 con 0x7f44e8102de0 2024-09-19T05:38:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.459+0000 7f44e57fa640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 0x7f44d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.459+0000 7f44e77fe640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 0x7f44d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.459+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f44d80cf8f0 con 0x7f44e8102de0 2024-09-19T05:38:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.460+0000 7f44e77fe640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 0x7f44d00a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44e81024d0 tx=0x7f44dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.464+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44d809c3c0 con 0x7f44e8102de0 2024-09-19T05:38:19.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.680+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44e8101f70 con 0x7f44e8102de0 2024-09-19T05:38:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.680+0000 7f44e57fa640 1 -- 172.21.15.38:0/4131538485 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f44d80a06c0 con 0x7f44e8102de0 2024-09-19T05:38:19.682 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:19.682 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":1,"btime":"2024-09-19T05:31:23:687807+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T05:38:19.682 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 1 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 msgr2=0x7f44d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 0x7f44d00a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f44e81024d0 tx=0x7f44dc00b390 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 msgr2=0x7f44e8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 shutdown_connections 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f44d00a3240 0x7f44d00a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e810c180 0x7f44e81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44e8103700 0x7f44e8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 --2- 172.21.15.38:0/4131538485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e8102de0 0x7f44e8100a40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 >> 172.21.15.38:0/4131538485 conn(0x7f44e80fc7a0 msgr2=0x7f44e80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:19.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 shutdown_connections 2024-09-19T05:38:19.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:19.682+0000 7f44ee50f640 1 -- 172.21.15.38:0/4131538485 wait complete. 2024-09-19T05:38:20.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:20 smithi038 ceph-mon[31227]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:20.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:20 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4131538485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:20.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:20 smithi179 ceph-mon[35781]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:20.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:20 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/4131538485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:20.947 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 05:38:20 smithi179 systemd[1]: Started Ceph mds.l for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:20.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:20 smithi047 ceph-mon[35327]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:20.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:20 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/4131538485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6832/2514682242,v1:172.21.15.179:6833/2514682242] up:boot 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: fsmap 1 up:standby 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:21.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:21 smithi038 ceph-mon[31227]: Deploying daemon mds.k on smithi047 2024-09-19T05:38:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6832/2514682242,v1:172.21.15.179:6833/2514682242] up:boot 2024-09-19T05:38:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: fsmap 1 up:standby 2024-09-19T05:38:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:21.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:21 smithi179 ceph-mon[35781]: Deploying daemon mds.k on smithi047 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6832/2514682242,v1:172.21.15.179:6833/2514682242] up:boot 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: fsmap 1 up:standby 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:21.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:21.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:21.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:21 smithi047 ceph-mon[35327]: Deploying daemon mds.k on smithi047 2024-09-19T05:38:22.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:22.672 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:23.082 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:22 smithi038 ceph-mon[31227]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:23.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:22 smithi047 ceph-mon[35327]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:23.116 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:22 smithi047 systemd[1]: Starting Ceph mds.k for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- 172.21.15.38:0/1014486722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf41036e0 msgr2=0x7fcaf4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 --2- 172.21.15.38:0/1014486722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf41036e0 0x7fcaf4107b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009930 tx=0x7fcae802a420 comp rx=0 tx=0).stop 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- 172.21.15.38:0/1014486722 shutdown_connections 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 --2- 172.21.15.38:0/1014486722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 0x7fcaf410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 --2- 172.21.15.38:0/1014486722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf41036e0 0x7fcaf4107b90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 --2- 172.21.15.38:0/1014486722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf4102d30 0x7fcaf4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- 172.21.15.38:0/1014486722 >> 172.21.15.38:0/1014486722 conn(0x7fcaf40fc7d0 msgr2=0x7fcaf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:23.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- 172.21.15.38:0/1014486722 shutdown_connections 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- 172.21.15.38:0/1014486722 wait complete. 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 Processor -- start 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.163+0000 7fcafc341640 1 -- start start 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf4102d30 0x7fcaf41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 0x7fcaf419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41120a0 con 0x7fcaf4102d30 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4111f20 con 0x7fcaf41036e0 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafc341640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4112220 con 0x7fcaf41082a0 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:44384/0 (socket says 172.21.15.38:44384) 2024-09-19T05:38:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 -- 172.21.15.38:0/226158531 learned_addr learned my addr 172.21.15.38:0/226158531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafa8de640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 0x7fcaf419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcafa0dd640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf4102d30 0x7fcaf41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 -- 172.21.15.38:0/226158531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 msgr2=0x7fcaf419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 0x7fcaf419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 -- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf4102d30 msgr2=0x7fcaf41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf4102d30 0x7fcaf41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 -- 172.21.15.38:0/226158531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf419d680 con 0x7fcaf41036e0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.164+0000 7fcaf98dc640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005940 tx=0x7fcae80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.165+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae803f030 con 0x7fcaf41036e0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.165+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf419d910 con 0x7fcaf41036e0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.165+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae80044c0 con 0x7fcaf41036e0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.165+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035070 con 0x7fcaf41036e0 2024-09-19T05:38:23.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.165+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41a5250 con 0x7fcaf41036e0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.166+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf406a850 con 0x7fcaf41036e0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.167+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcae8042030 con 0x7fcaf41036e0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.168+0000 7fcaf37fe640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 0x7fcadc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.168+0000 7fcafa0dd640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 0x7fcadc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.168+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcae80efb20 con 0x7fcaf41036e0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.168+0000 7fcafa0dd640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 0x7fcadc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bd0 tx=0x7fcaec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.171+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcae80bc5f0 con 0x7fcaf41036e0 2024-09-19T05:38:23.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:22 smithi179 ceph-mon[35781]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:23.378 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 podman[64246]: 2024-09-19 05:38:23.114617974 +0000 UTC m=+0.111543512 container create faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T05:38:23.378 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 podman[64246]: 2024-09-19 05:38:23.019099056 +0000 UTC m=+0.016024563 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:23.378 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 podman[64246]: 2024-09-19 05:38:23.350517059 +0000 UTC m=+0.347442575 container init faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:38:23.378 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 podman[64246]: 2024-09-19 05:38:23.353425117 +0000 UTC m=+0.350350614 container start faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T05:38:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.382+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf4103110 con 0x7fcaf41036e0 2024-09-19T05:38:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.383+0000 7fcaf37fe640 1 -- 172.21.15.38:0/226158531 <== mon.2 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fcaf4103110 con 0x7fcaf41036e0 2024-09-19T05:38:23.385 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:23.385 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"btime":"2024-09-19T05:38:20:607397+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:38:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 msgr2=0x7fcadc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 0x7fcadc0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bd0 tx=0x7fcaec005f90 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 msgr2=0x7fcaf4198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005940 tx=0x7fcae80040c0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 shutdown_connections 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcadc0a3240 0x7fcadc0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaf41082a0 0x7fcaf419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf41036e0 0x7fcaf4198c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 --2- 172.21.15.38:0/226158531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcaf4102d30 0x7fcaf41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 >> 172.21.15.38:0/226158531 conn(0x7fcaf40fc7d0 msgr2=0x7fcaf4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:23.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 shutdown_connections 2024-09-19T05:38:23.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:23.385+0000 7fcafc341640 1 -- 172.21.15.38:0/226158531 wait complete. 2024-09-19T05:38:23.726 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mds[64266]: -- 172.21.15.47:0/1422001134 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575c3333860 con 0x5575c3311400 2024-09-19T05:38:23.726 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k[64262]: starting mds.k at 2024-09-19T05:38:23.726 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 bash[64246]: faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f 2024-09-19T05:38:23.726 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 05:38:23 smithi047 systemd[1]: Started Ceph mds.k for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:24.138 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/226158531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:24.139 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:23 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:24.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/226158531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:24.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:24.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:24.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:23 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/226158531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:24.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:23 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:24.794 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:24 smithi047 ceph-mon[35327]: Deploying daemon mds.j on smithi038 2024-09-19T05:38:24.794 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:24 smithi047 ceph-mon[35327]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:24.794 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:24 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6832/418551929,v1:172.21.15.47:6833/418551929] up:boot 2024-09-19T05:38:24.794 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:24 smithi047 ceph-mon[35327]: fsmap 2 up:standby 2024-09-19T05:38:24.795 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:24 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:38:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:24 smithi038 ceph-mon[31227]: Deploying daemon mds.j on smithi038 2024-09-19T05:38:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:24 smithi038 ceph-mon[31227]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:24 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6832/418551929,v1:172.21.15.47:6833/418551929] up:boot 2024-09-19T05:38:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:24 smithi038 ceph-mon[31227]: fsmap 2 up:standby 2024-09-19T05:38:25.071 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:24 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:38:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:24 smithi179 ceph-mon[35781]: Deploying daemon mds.j on smithi038 2024-09-19T05:38:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:24 smithi179 ceph-mon[35781]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:24 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6832/418551929,v1:172.21.15.47:6833/418551929] up:boot 2024-09-19T05:38:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:24 smithi179 ceph-mon[35781]: fsmap 2 up:standby 2024-09-19T05:38:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:24 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:38:26.122 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:25 smithi038 systemd[1]: Starting Ceph mds.j for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:26.169 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:26.394 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 podman[68750]: 2024-09-19 05:38:26.023685121 +0000 UTC m=+0.016140305 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:26.394 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 podman[68750]: 2024-09-19 05:38:26.131950779 +0000 UTC m=+0.124405940 container create 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:38:26.612 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:26.736 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 podman[68750]: 2024-09-19 05:38:26.393571292 +0000 UTC m=+0.386026468 container init 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T05:38:26.737 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 podman[68750]: 2024-09-19 05:38:26.396694033 +0000 UTC m=+0.389149194 container start 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T05:38:26.737 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mds[68800]: -- 172.21.15.38:0/65770995 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55674576b860 con 0x556745748800 2024-09-19T05:38:26.737 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j[68783]: starting mds.j at 2024-09-19T05:38:26.737 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 bash[68750]: 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 2024-09-19T05:38:26.737 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 05:38:26 smithi038 systemd[1]: Started Ceph mds.j for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:27.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:26 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:27.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1514783465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0100e20 msgr2=0x7fa3a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:27.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1514783465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0100e20 0x7fa3a0101200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa390009900 tx=0x7fa39002a480 comp rx=0 tx=0).stop 2024-09-19T05:38:27.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1514783465 shutdown_connections 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1514783465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0102ef0 0x7fa3a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1514783465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 0x7fa3a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1514783465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0100e20 0x7fa3a0101200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1514783465 >> 172.21.15.38:0/1514783465 conn(0x7fa3a00fc7d0 msgr2=0x7fa3a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1514783465 shutdown_connections 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.116+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1514783465 wait complete. 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 Processor -- start 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 -- start start 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0100e20 0x7fa3a0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 0x7fa3a01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0110fa0 con 0x7fa3a0102ef0 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0110e20 con 0x7fa3a0100e20 2024-09-19T05:38:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa3a5a93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0111120 con 0x7fa3a0102570 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40448/0 (socket says 172.21.15.38:40448) 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa39ffff640 1 -- 172.21.15.38:0/1751005120 learned_addr learned my addr 172.21.15.38:0/1751005120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa396ffd640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 0x7fa3a01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.117+0000 7fa39f7fe640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0100e20 0x7fa3a0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 -- 172.21.15.38:0/1751005120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 msgr2=0x7fa3a01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 0x7fa3a01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 -- 172.21.15.38:0/1751005120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0100e20 msgr2=0x7fa3a0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0100e20 0x7fa3a0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 -- 172.21.15.38:0/1751005120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a01a4930 con 0x7fa3a0102ef0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39ffff640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa39800cce0 tx=0x7fa39800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800f450 con 0x7fa3a0102ef0 2024-09-19T05:38:27.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01a4b00 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa39800fad0 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398018b60 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.118+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a5010 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.119+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a0101200 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.120+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa39800fc80 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.121+0000 7fa39d7fa640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 0x7fa38c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.121+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa3980cf9d0 con 0x7fa3a0102ef0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.121+0000 7fa39f7fe640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 0x7fa38c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.121+0000 7fa39f7fe640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 0x7fa38c0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa390005ab0 tx=0x7fa390005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:27.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.125+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa398014030 con 0x7fa3a0102ef0 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:26 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:27.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:26 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:27.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.350+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a006a560 con 0x7fa3a0102ef0 2024-09-19T05:38:27.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.351+0000 7fa39d7fa640 1 -- 172.21.15.38:0/1751005120 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fa3a006a560 con 0x7fa3a0102ef0 2024-09-19T05:38:27.354 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:27.354 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"btime":"2024-09-19T05:38:26:754434+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:38:27.354 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-09-19T05:38:27.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 msgr2=0x7fa38c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 0x7fa38c0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa390005ab0 tx=0x7fa390005a20 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 msgr2=0x7fa3a01a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa39800cce0 tx=0x7fa39800a620 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 shutdown_connections 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa38c0a3240 0x7fa38c0a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3a0102ef0 0x7fa3a01a43f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a0102570 0x7fa3a01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 --2- 172.21.15.38:0/1751005120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3a0100e20 0x7fa3a0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 >> 172.21.15.38:0/1751005120 conn(0x7fa3a00fc7d0 msgr2=0x7fa3a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 shutdown_connections 2024-09-19T05:38:27.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:27.353+0000 7fa3a5a93640 1 -- 172.21.15.38:0/1751005120 wait complete. 2024-09-19T05:38:28.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:27 smithi038 ceph-mon[31227]: Deploying daemon mds.d on smithi038 2024-09-19T05:38:28.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:27 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6834/1823768256,v1:172.21.15.38:6835/1823768256] up:boot 2024-09-19T05:38:28.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:27 smithi038 ceph-mon[31227]: fsmap 3 up:standby 2024-09-19T05:38:28.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:27 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:38:28.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:27 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1751005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:27 smithi179 ceph-mon[35781]: Deploying daemon mds.d on smithi038 2024-09-19T05:38:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:27 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6834/1823768256,v1:172.21.15.38:6835/1823768256] up:boot 2024-09-19T05:38:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:27 smithi179 ceph-mon[35781]: fsmap 3 up:standby 2024-09-19T05:38:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:27 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:38:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:27 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1751005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:28.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:27 smithi047 ceph-mon[35327]: Deploying daemon mds.d on smithi038 2024-09-19T05:38:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:27 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6834/1823768256,v1:172.21.15.38:6835/1823768256] up:boot 2024-09-19T05:38:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:27 smithi047 ceph-mon[35327]: fsmap 3 up:standby 2024-09-19T05:38:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:27 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:38:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:27 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1751005120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:29.089 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:28 smithi038 ceph-mon[31227]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:28 smithi179 ceph-mon[35781]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:29.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:28 smithi047 ceph-mon[35327]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:29.803 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:29 smithi038 systemd[1]: Starting Ceph mds.d for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:30.094 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:29 smithi038 podman[69167]: 2024-09-19 05:38:29.704370048 +0000 UTC m=+0.016502442 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:30.094 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:29 smithi038 podman[69167]: 2024-09-19 05:38:29.84693956 +0000 UTC m=+0.159071934 container create 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-19T05:38:30.094 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 podman[69167]: 2024-09-19 05:38:30.093247402 +0000 UTC m=+0.405379792 container init 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-19T05:38:30.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:30.388 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:30.394 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 podman[69167]: 2024-09-19 05:38:30.096344694 +0000 UTC m=+0.408477076 container start 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T05:38:30.394 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 ceph-mds[69186]: -- 172.21.15.38:0/2519210218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560dee0bf860 con 0x560dee09c800 2024-09-19T05:38:30.394 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: starting mds.d at 2024-09-19T05:38:30.394 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 bash[69167]: 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc 2024-09-19T05:38:30.394 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:38:30 smithi038 systemd[1]: Started Ceph mds.d for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 -- 172.21.15.38:0/3611344873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 msgr2=0x7f06c0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 --2- 172.21.15.38:0/3611344873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c0100f10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009900 tx=0x7f06bc02a470 comp rx=0 tx=0).stop 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 -- 172.21.15.38:0/3611344873 shutdown_connections 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 --2- 172.21.15.38:0/3611344873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 --2- 172.21.15.38:0/3611344873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 0x7f06c0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 --2- 172.21.15.38:0/3611344873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c0100f10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.879+0000 7f06c76f2640 1 -- 172.21.15.38:0/3611344873 >> 172.21.15.38:0/3611344873 conn(0x7f06c00fc7a0 msgr2=0x7f06c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- 172.21.15.38:0/3611344873 shutdown_connections 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- 172.21.15.38:0/3611344873 wait complete. 2024-09-19T05:38:30.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 Processor -- start 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- start start 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 0x7f06c01a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0118550 con 0x7f06c0100b30 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f06c01183d0 con 0x7f06c0105f40 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c76f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06c01186d0 con 0x7f06c0101450 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c548e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c548e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42888/0 (socket says 172.21.15.38:42888) 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.880+0000 7f06c548e640 1 -- 172.21.15.38:0/2787702350 learned_addr learned my addr 172.21.15.38:0/2787702350 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c5c8f640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:30.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c4c8d640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 0x7f06c01a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 -- 172.21.15.38:0/2787702350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 msgr2=0x7f06c01a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 0x7f06c01a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 -- 172.21.15.38:0/2787702350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 msgr2=0x7f06c019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 -- 172.21.15.38:0/2787702350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c019ef80 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c5c8f640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c548e640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06bc006010 tx=0x7f06bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc03f030 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c019f210 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06bc003ce0 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc035070 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.881+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01ab4c0 con 0x7f06c0100b30 2024-09-19T05:38:30.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.882+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c006b070 con 0x7f06c0100b30 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.884+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f06bc042030 con 0x7f06c0100b30 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.885+0000 7f06b67fc640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 0x7f06ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.885+0000 7f06c4c8d640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 0x7f06ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.885+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f06bc0efed0 con 0x7f06c0100b30 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.885+0000 7f06c4c8d640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 0x7f06ac0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005fd0 tx=0x7f06b0005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:30.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:30.888+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06bc0bd0b0 con 0x7f06c0100b30 2024-09-19T05:38:31.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.099+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c01ab690 con 0x7f06c0100b30 2024-09-19T05:38:31.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.100+0000 7f06b67fc640 1 -- 172.21.15.38:0/2787702350 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f06bc0bc9b0 con 0x7f06c0100b30 2024-09-19T05:38:31.103 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:31.103 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"btime":"2024-09-19T05:38:30:774928+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:38:31.103 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 5 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 msgr2=0x7f06ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 0x7f06ac0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06b0005fd0 tx=0x7f06b0005f60 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 msgr2=0x7f06c01a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06bc006010 tx=0x7f06bc0042f0 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 shutdown_connections 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f06ac0a3240 0x7f06ac0a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06c0105f40 0x7f06c019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06c0101450 0x7f06c01a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 --2- 172.21.15.38:0/2787702350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c0100b30 0x7f06c01a4660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 >> 172.21.15.38:0/2787702350 conn(0x7f06c00fc7a0 msgr2=0x7f06c01038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:31.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 shutdown_connections 2024-09-19T05:38:31.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:31.102+0000 7f06c76f2640 1 -- 172.21.15.38:0/2787702350 wait complete. 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: Deploying daemon mds.e on smithi047 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6836/4064878755,v1:172.21.15.38:6837/4064878755] up:boot 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: fsmap 4 up:standby 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:38:31.395 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2787702350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: Deploying daemon mds.e on smithi047 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6836/4064878755,v1:172.21.15.38:6837/4064878755] up:boot 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: fsmap 4 up:standby 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:38:31.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:31 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2787702350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: Deploying daemon mds.e on smithi047 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6836/4064878755,v1:172.21.15.38:6837/4064878755] up:boot 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: fsmap 4 up:standby 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:38:31.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:31 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2787702350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:32.454 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 systemd[1]: Starting Ceph mds.e for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:32.726 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 podman[64563]: 2024-09-19 05:38:32.355973271 +0000 UTC m=+0.015970910 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:32.726 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 podman[64563]: 2024-09-19 05:38:32.453313625 +0000 UTC m=+0.113311269 container create 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T05:38:32.727 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 podman[64563]: 2024-09-19 05:38:32.672859946 +0000 UTC m=+0.332857587 container init 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T05:38:32.727 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 podman[64563]: 2024-09-19 05:38:32.67553077 +0000 UTC m=+0.335528408 container start 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:38:32.727 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 ceph-mds[64582]: -- 172.21.15.47:0/2979067382 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56291c777860 con 0x56291c754800 2024-09-19T05:38:32.727 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: starting mds.e at 2024-09-19T05:38:33.226 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 bash[64563]: 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 2024-09-19T05:38:33.226 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:38:32 smithi047 systemd[1]: Started Ceph mds.e for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:33.514 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:33 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:33 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:33 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:33.941 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:34.212 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:34.557 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:34 smithi179 ceph-mon[35781]: Deploying daemon mds.f on smithi179 2024-09-19T05:38:34.557 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:34 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6834/3758113926,v1:172.21.15.47:6835/3758113926] up:boot 2024-09-19T05:38:34.557 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:34 smithi179 ceph-mon[35781]: fsmap 5 up:standby 2024-09-19T05:38:34.557 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:34 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:38:34.569 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:34 smithi038 ceph-mon[31227]: Deploying daemon mds.f on smithi179 2024-09-19T05:38:34.569 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:34 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6834/3758113926,v1:172.21.15.47:6835/3758113926] up:boot 2024-09-19T05:38:34.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:34 smithi038 ceph-mon[31227]: fsmap 5 up:standby 2024-09-19T05:38:34.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:34 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:38:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:34 smithi047 ceph-mon[35327]: Deploying daemon mds.f on smithi179 2024-09-19T05:38:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:34 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6834/3758113926,v1:172.21.15.47:6835/3758113926] up:boot 2024-09-19T05:38:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:34 smithi047 ceph-mon[35327]: fsmap 5 up:standby 2024-09-19T05:38:34.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:34 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:38:34.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 -- 172.21.15.38:0/1887959466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c103660 msgr2=0x7f2d0c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:34.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 --2- 172.21.15.38:0/1887959466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c103660 0x7f2d0c10bb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009900 tx=0x7f2d0002a480 comp rx=0 tx=0).stop 2024-09-19T05:38:34.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 -- 172.21.15.38:0/1887959466 shutdown_connections 2024-09-19T05:38:34.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 --2- 172.21.15.38:0/1887959466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 0x7f2d0c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:34.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 --2- 172.21.15.38:0/1887959466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c103660 0x7f2d0c10bb50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 --2- 172.21.15.38:0/1887959466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c102d40 0x7f2d0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 -- 172.21.15.38:0/1887959466 >> 172.21.15.38:0/1887959466 conn(0x7f2d0c0fc7a0 msgr2=0x7f2d0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 -- 172.21.15.38:0/1887959466 shutdown_connections 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.793+0000 7f2d12817640 1 -- 172.21.15.38:0/1887959466 wait complete. 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 Processor -- start 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 -- start start 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c103660 0x7f2d0c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 0x7f2d0c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c111f70 con 0x7f2d0c102d40 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c111df0 con 0x7f2d0c10c170 2024-09-19T05:38:34.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.794+0000 7f2d12817640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1120f0 con 0x7f2d0c103660 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42904/0 (socket says 172.21.15.38:42904) 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 -- 172.21.15.38:0/138428842 learned_addr learned my addr 172.21.15.38:0/138428842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0b7fe640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c103660 0x7f2d0c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d10db4640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 0x7f2d0c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 -- 172.21.15.38:0/138428842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c103660 msgr2=0x7f2d0c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c103660 0x7f2d0c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 -- 172.21.15.38:0/138428842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 msgr2=0x7f2d0c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 0x7f2d0c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 -- 172.21.15.38:0/138428842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c1a6910 con 0x7f2d0c102d40 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d0bfff640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00e9f0 tx=0x7f2cfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00cd80 con 0x7f2d0c102d40 2024-09-19T05:38:34.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc005500 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c1a6c00 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc010690 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.795+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c1a70c0 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.796+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c103120 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.798+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2cfc0046c0 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.798+0000 7f2d097fa640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 0x7f2cf40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.798+0000 7f2d0b7fe640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 0x7f2cf40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.798+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2cfc0cf890 con 0x7f2d0c102d40 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.799+0000 7f2d0b7fe640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 0x7f2cf40a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009230 tx=0x7f2d000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:34.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:34.803+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2cfc09ca50 con 0x7f2d0c102d40 2024-09-19T05:38:34.943 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:34 smithi179 systemd[1]: Starting Ceph mds.f for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.013+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d0c02d7c0 con 0x7f2d0c102d40 2024-09-19T05:38:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.013+0000 7f2d097fa640 1 -- 172.21.15.38:0/138428842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f2cfc09c350 con 0x7f2d0c102d40 2024-09-19T05:38:35.016 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:35.016 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"btime":"2024-09-19T05:38:33:257216+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:38:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2024-09-19T05:38:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.015+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 msgr2=0x7f2cf40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.015+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 0x7f2cf40a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009230 tx=0x7f2d000058d0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.015+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 msgr2=0x7f2d0c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.015+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00e9f0 tx=0x7f2cfc00edf0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 shutdown_connections 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2cf40a3240 0x7f2cf40a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d0c10c170 0x7f2d0c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c103660 0x7f2d0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 --2- 172.21.15.38:0/138428842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d0c102d40 0x7f2d0c1009a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 >> 172.21.15.38:0/138428842 conn(0x7f2d0c0fc7a0 msgr2=0x7f2d0c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 shutdown_connections 2024-09-19T05:38:35.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:35.016+0000 7f2d12817640 1 -- 172.21.15.38:0/138428842 wait complete. 2024-09-19T05:38:35.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:34 smithi179 podman[65693]: 2024-09-19 05:38:34.844203759 +0000 UTC m=+0.015907299 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:35.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:34 smithi179 podman[65693]: 2024-09-19 05:38:34.949635258 +0000 UTC m=+0.121338830 container create a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T05:38:35.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 podman[65693]: 2024-09-19 05:38:35.185801764 +0000 UTC m=+0.357505304 container init a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T05:38:35.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 podman[65693]: 2024-09-19 05:38:35.18851374 +0000 UTC m=+0.360217271 container start a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T05:38:35.631 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:35 smithi038 ceph-mon[31227]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:35.631 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:35 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/138428842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:35 smithi179 ceph-mon[35781]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:35 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/138428842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:35.697 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 ceph-mds[65712]: -- 172.21.15.179:0/1237383068 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564604203860 con 0x5646041e0800 2024-09-19T05:38:35.697 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: starting mds.f at 2024-09-19T05:38:35.698 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 bash[65693]: a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 2024-09-19T05:38:35.698 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:38:35 smithi179 systemd[1]: Started Ceph mds.f for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:35 smithi047 ceph-mon[35327]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:35 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/138428842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:boot 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: fsmap 6 up:standby 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:36.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:36 smithi038 ceph-mon[31227]: Deploying daemon mds.c on smithi179 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:boot 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: fsmap 6 up:standby 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:36.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:36 smithi179 ceph-mon[35781]: Deploying daemon mds.c on smithi179 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:boot 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: fsmap 6 up:standby 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:36.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:36.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:36.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:36.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:36 smithi047 ceph-mon[35327]: Deploying daemon mds.c on smithi179 2024-09-19T05:38:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:37 smithi179 ceph-mon[35781]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:37 smithi038 ceph-mon[31227]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:37.660 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:37 smithi047 ceph-mon[35327]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:37.940 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:38.197 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 05:38:37 smithi179 systemd[1]: Started Ceph mds.c for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 -- 172.21.15.38:0/2808451286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578102790 msgr2=0x7fd5781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2808451286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578102790 0x7fd5781067e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd564009900 tx=0x7fd56402a410 comp rx=0 tx=0).stop 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 -- 172.21.15.38:0/2808451286 shutdown_connections 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2808451286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578106f10 0x7fd57810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2808451286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578102790 0x7fd5781067e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2808451286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578101de0 0x7fd5781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 -- 172.21.15.38:0/2808451286 >> 172.21.15.38:0/2808451286 conn(0x7fd5780fd8c0 msgr2=0x7fd5780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 -- 172.21.15.38:0/2808451286 shutdown_connections 2024-09-19T05:38:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.464+0000 7fd57ed64640 1 -- 172.21.15.38:0/2808451286 wait complete. 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 Processor -- start 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 -- start start 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578102790 0x7fd578074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 0x7fd578074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd578111f70 con 0x7fd578101de0 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd578111df0 con 0x7fd578106f10 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57ed64640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd5781120f0 con 0x7fd578102790 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59634/0 (socket says 172.21.15.38:59634) 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 -- 172.21.15.38:0/2681371502 learned_addr learned my addr 172.21.15.38:0/2681371502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57d301640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 0x7fd578074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd577fff640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578102790 0x7fd578074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 -- 172.21.15.38:0/2681371502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578102790 msgr2=0x7fd578074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578102790 0x7fd578074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 -- 172.21.15.38:0/2681371502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 msgr2=0x7fd578074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 0x7fd578074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57cb00640 1 -- 172.21.15.38:0/2681371502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5781a6990 con 0x7fd578101de0 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.465+0000 7fd57d301640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 0x7fd578074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd57cb00640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e9f0 tx=0x7fd56c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00cd80 con 0x7fd578101de0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5781a6c80 con 0x7fd578101de0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56c010910 con 0x7fd578101de0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0188f0 con 0x7fd578101de0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.466+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5781a7190 con 0x7fd578101de0 2024-09-19T05:38:38.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.467+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5781021c0 con 0x7fd578101de0 2024-09-19T05:38:38.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.468+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd56c010ac0 con 0x7fd578101de0 2024-09-19T05:38:38.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.469+0000 7fd575ffb640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 0x7fd5680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:38.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.469+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd56c0cf8c0 con 0x7fd578101de0 2024-09-19T05:38:38.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.469+0000 7fd577fff640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 0x7fd5680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:38.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.469+0000 7fd577fff640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 0x7fd5680a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd564009230 tx=0x7fd5640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.473+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd56c09c390 con 0x7fd578101de0 2024-09-19T05:38:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.686+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd57810a150 con 0x7fd578101de0 2024-09-19T05:38:38.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.687+0000 7fd575ffb640 1 -- 172.21.15.38:0/2681371502 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7fd56c0a0690 con 0x7fd578101de0 2024-09-19T05:38:38.690 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:38.690 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":8,"btime":"2024-09-19T05:38:38:340459+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T05:38:38.690 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 8 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 msgr2=0x7fd5680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 0x7fd5680a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd564009230 tx=0x7fd5640058d0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 msgr2=0x7fd5780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e9f0 tx=0x7fd56c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 shutdown_connections 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd5680a3240 0x7fd5680a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd578106f10 0x7fd578074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd578102790 0x7fd578074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 --2- 172.21.15.38:0/2681371502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd578101de0 0x7fd5780744f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 >> 172.21.15.38:0/2681371502 conn(0x7fd5780fd8c0 msgr2=0x7fd5780fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 shutdown_connections 2024-09-19T05:38:38.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:38.689+0000 7fd57ed64640 1 -- 172.21.15.38:0/2681371502 wait complete. 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: Deploying daemon mds.h on smithi047 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:boot 2024-09-19T05:38:38.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: fsmap 7 up:standby 2024-09-19T05:38:38.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:38:38.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:38 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2681371502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:39.204 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:39.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: Deploying daemon mds.h on smithi047 2024-09-19T05:38:39.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:boot 2024-09-19T05:38:39.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: fsmap 7 up:standby 2024-09-19T05:38:39.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:38:39.205 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:38 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2681371502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: Deploying daemon mds.h on smithi047 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:boot 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: fsmap 7 up:standby 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:38:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:38 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2681371502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:40.188 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:39 smithi047 systemd[1]: Starting Ceph mds.h for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:40.439 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 podman[64868]: 2024-09-19 05:38:40.089267466 +0000 UTC m=+0.015807607 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:40.439 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 podman[64868]: 2024-09-19 05:38:40.192627839 +0000 UTC m=+0.119168007 container create 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T05:38:40.439 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 podman[64868]: 2024-09-19 05:38:40.411940655 +0000 UTC m=+0.338480796 container init 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T05:38:40.439 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 podman[64868]: 2024-09-19 05:38:40.41460672 +0000 UTC m=+0.341146855 container start 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T05:38:40.725 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mds[64887]: -- 172.21.15.47:0/817217289 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625e351b860 con 0x5625e34f8800 2024-09-19T05:38:40.726 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: starting mds.h at 2024-09-19T05:38:40.726 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 bash[64868]: 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 2024-09-19T05:38:40.726 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:38:40 smithi047 systemd[1]: Started Ceph mds.h for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:40 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:41.297 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:40 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:41.328 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:41.359 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:41.359 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.359 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.359 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:41.360 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:41.360 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:41.360 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:40 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:41.603 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:42.018 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:41 smithi038 ceph-mon[31227]: Deploying daemon mds.i on smithi179 2024-09-19T05:38:42.019 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:41 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:boot 2024-09-19T05:38:42.019 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:41 smithi038 ceph-mon[31227]: fsmap 8 up:standby 2024-09-19T05:38:42.019 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:41 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- 172.21.15.38:0/164826361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14102570 msgr2=0x7ffb141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 --2- 172.21.15.38:0/164826361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14102570 0x7ffb141029b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009900 tx=0x7ffb0802a480 comp rx=0 tx=0).stop 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- 172.21.15.38:0/164826361 shutdown_connections 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 --2- 172.21.15.38:0/164826361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102ef0 0x7ffb1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 --2- 172.21.15.38:0/164826361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14102570 0x7ffb141029b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 --2- 172.21.15.38:0/164826361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14100e20 0x7ffb14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- 172.21.15.38:0/164826361 >> 172.21.15.38:0/164826361 conn(0x7ffb140fc7d0 msgr2=0x7ffb140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- 172.21.15.38:0/164826361 shutdown_connections 2024-09-19T05:38:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- 172.21.15.38:0/164826361 wait complete. 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 Processor -- start 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.100+0000 7ffb1a541640 1 -- start start 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102570 0x7ffb14198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14102ef0 0x7ffb1419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141120a0 con 0x7ffb14100e20 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14111f20 con 0x7ffb14102570 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb1a541640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14112220 con 0x7ffb14102ef0 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59646/0 (socket says 172.21.15.38:59646) 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 -- 172.21.15.38:0/2214410725 learned_addr learned my addr 172.21.15.38:0/2214410725 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb18ade640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14102ef0 0x7ffb1419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb137fe640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102570 0x7ffb14198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 -- 172.21.15.38:0/2214410725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14102ef0 msgr2=0x7ffb1419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14102ef0 0x7ffb1419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 -- 172.21.15.38:0/2214410725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102570 msgr2=0x7ffb14198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102570 0x7ffb14198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 -- 172.21.15.38:0/2214410725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1419d570 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.101+0000 7ffb13fff640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b4a0 tx=0x7ffb0400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.102+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04014070 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.102+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb04010af0 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.102+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1419d860 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.102+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04017d10 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.102+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a5140 con 0x7ffb14100e20 2024-09-19T05:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.103+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb14101200 con 0x7ffb14100e20 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.104+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ffb04017070 con 0x7ffb14100e20 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.104+0000 7ffb117fa640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 0x7ffafc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.105+0000 7ffb137fe640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 0x7ffafc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.105+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ffb040ce8e0 con 0x7ffb14100e20 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.105+0000 7ffb137fe640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 0x7ffafc0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb140ffdc0 tx=0x7ffb08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:42.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.109+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffb0409ba00 con 0x7ffb14100e20 2024-09-19T05:38:42.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:41 smithi047 ceph-mon[35327]: Deploying daemon mds.i on smithi179 2024-09-19T05:38:42.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:41 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:boot 2024-09-19T05:38:42.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:41 smithi047 ceph-mon[35327]: fsmap 8 up:standby 2024-09-19T05:38:42.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:41 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:38:42.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.323+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb14109090 con 0x7ffb14100e20 2024-09-19T05:38:42.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.324+0000 7ffb117fa640 1 -- 172.21.15.38:0/2214410725 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7ffb0409b300 con 0x7ffb14100e20 2024-09-19T05:38:42.326 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:42.326 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-19T05:38:40:955709+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T05:38:42.326 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 msgr2=0x7ffafc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 0x7ffafc0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb140ffdc0 tx=0x7ffb08005a20 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 msgr2=0x7ffb141985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b4a0 tx=0x7ffb0400b960 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 shutdown_connections 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ffafc0a3240 0x7ffafc0a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14102ef0 0x7ffb1419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb14102570 0x7ffb14198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 --2- 172.21.15.38:0/2214410725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb14100e20 0x7ffb141985c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 >> 172.21.15.38:0/2214410725 conn(0x7ffb140fc7d0 msgr2=0x7ffb140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:42.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 shutdown_connections 2024-09-19T05:38:42.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:42.326+0000 7ffb1a541640 1 -- 172.21.15.38:0/2214410725 wait complete. 2024-09-19T05:38:42.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:41 smithi179 ceph-mon[35781]: Deploying daemon mds.i on smithi179 2024-09-19T05:38:42.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:41 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:boot 2024-09-19T05:38:42.347 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:41 smithi179 ceph-mon[35781]: fsmap 8 up:standby 2024-09-19T05:38:42.347 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:41 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:38:42.697 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:42 smithi179 systemd[1]: Starting Ceph mds.i for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:43.014 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:42 smithi179 ceph-mon[35781]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:43.015 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:42 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2214410725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:43.015 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:42 smithi179 podman[66297]: 2024-09-19 05:38:42.648410812 +0000 UTC m=+0.015871217 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:43.015 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:42 smithi179 podman[66297]: 2024-09-19 05:38:42.777679607 +0000 UTC m=+0.145140041 container create 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:38:43.015 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 podman[66297]: 2024-09-19 05:38:43.01372096 +0000 UTC m=+0.381181353 container init 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T05:38:43.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:42 smithi038 ceph-mon[31227]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:43.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:42 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2214410725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:43.447 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 podman[66297]: 2024-09-19 05:38:43.016573425 +0000 UTC m=+0.384033826 container start 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T05:38:43.447 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 ceph-mds[66316]: -- 172.21.15.179:0/2100322484 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9ef231860 con 0x55b9ef20f000 2024-09-19T05:38:43.447 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: starting mds.i at 2024-09-19T05:38:43.447 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 bash[66297]: 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd 2024-09-19T05:38:43.448 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:38:43 smithi179 systemd[1]: Started Ceph mds.i for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:42 smithi047 ceph-mon[35327]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:42 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2214410725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: Deploying daemon mds.g on smithi038 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:boot 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: fsmap 9 up:standby 2024-09-19T05:38:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:44 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: Deploying daemon mds.g on smithi038 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:boot 2024-09-19T05:38:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: fsmap 9 up:standby 2024-09-19T05:38:44.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:44 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: Deploying daemon mds.g on smithi038 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:boot 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: fsmap 9 up:standby 2024-09-19T05:38:44.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:44 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:38:44.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:45.144 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 systemd[1]: Starting Ceph mds.g for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:45 smithi179 ceph-mon[35781]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:45.448 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:45 smithi038 ceph-mon[31227]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:45.449 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 podman[69958]: 2024-09-19 05:38:45.205581621 +0000 UTC m=+0.017016913 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:45.449 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 podman[69958]: 2024-09-19 05:38:45.325267411 +0000 UTC m=+0.136702695 container create 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:38:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:45 smithi047 ceph-mon[35327]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:45.840 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 podman[69958]: 2024-09-19 05:38:45.59530895 +0000 UTC m=+0.406744243 container init 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 podman[69958]: 2024-09-19 05:38:45.59878867 +0000 UTC m=+0.410223963 container start 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 ceph-mds[69989]: -- 172.21.15.38:0/905953656 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f16f871c20 con 0x55f16f84f400 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: starting mds.g at 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 bash[69958]: 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d 2024-09-19T05:38:45.894 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:38:45 smithi038 systemd[1]: Started Ceph mds.g for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 -- 172.21.15.38:0/1726652117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 msgr2=0x7ff4701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 --2- 172.21.15.38:0/1726652117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701021c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff460009900 tx=0x7ff46002a410 comp rx=0 tx=0).stop 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 -- 172.21.15.38:0/1726652117 shutdown_connections 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 --2- 172.21.15.38:0/1726652117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff470106f10 0x7ff47010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 --2- 172.21.15.38:0/1726652117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 0x7ff4701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 --2- 172.21.15.38:0/1726652117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701021c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 -- 172.21.15.38:0/1726652117 >> 172.21.15.38:0/1726652117 conn(0x7ff4700fd8c0 msgr2=0x7ff4700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 -- 172.21.15.38:0/1726652117 shutdown_connections 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.340+0000 7ff476783640 1 -- 172.21.15.38:0/1726652117 wait complete. 2024-09-19T05:38:46.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 Processor -- start 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 -- start start 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 0x7ff470198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff470106f10 0x7ff47019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff470111f70 con 0x7ff470101de0 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff470111df0 con 0x7ff470106f10 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff476783640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4701120f0 con 0x7ff470102790 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59676/0 (socket says 172.21.15.38:59676) 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.341+0000 7ff46ffff640 1 -- 172.21.15.38:0/3497108373 learned_addr learned my addr 172.21.15.38:0/3497108373 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46f7fe640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 0x7ff470198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:46.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 -- 172.21.15.38:0/3497108373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 msgr2=0x7ff470198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 0x7ff470198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 -- 172.21.15.38:0/3497108373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff470106f10 msgr2=0x7ff47019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff470106f10 0x7ff47019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 -- 172.21.15.38:0/3497108373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47019d480 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46ffff640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff4600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46003f030 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47019d710 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff460003ce0 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460035070 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.342+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701a5050 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.343+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4701021c0 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.344+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff460042030 con 0x7ff470101de0 2024-09-19T05:38:46.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.345+0000 7ff46d7fa640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 0x7ff4580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:46.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.345+0000 7ff46f7fe640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 0x7ff4580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:46.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.345+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ff4600efb50 con 0x7ff470101de0 2024-09-19T05:38:46.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.345+0000 7ff46f7fe640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 0x7ff4580a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff464005fd0 tx=0x7ff464005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:46.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.349+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4600bcdd0 con 0x7ff470101de0 2024-09-19T05:38:46.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.574+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff470199830 con 0x7ff470101de0 2024-09-19T05:38:46.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.575+0000 7ff46d7fa640 1 -- 172.21.15.38:0/3497108373 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7ff4600bc6d0 con 0x7ff470101de0 2024-09-19T05:38:46.577 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:46.577 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"btime":"2024-09-19T05:38:46:197125+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:38:46.577 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-09-19T05:38:46.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 msgr2=0x7ff4580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:46.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 0x7ff4580a5700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff464005fd0 tx=0x7ff464005850 comp rx=0 tx=0).stop 2024-09-19T05:38:46.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 msgr2=0x7ff4701984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:46.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff4600042f0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 shutdown_connections 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4580a3240 0x7ff4580a5700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff470106f10 0x7ff47019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff470102790 0x7ff470198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 --2- 172.21.15.38:0/3497108373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff470101de0 0x7ff4701984d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.577+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 >> 172.21.15.38:0/3497108373 conn(0x7ff4700fd8c0 msgr2=0x7ff4700fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.578+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 shutdown_connections 2024-09-19T05:38:46.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:46.578+0000 7ff476783640 1 -- 172.21.15.38:0/3497108373 wait complete. 2024-09-19T05:38:46.863 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: Deploying daemon mds.a on smithi038 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6838/2963619728,v1:172.21.15.38:6839/2963619728] up:boot 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: fsmap 10 up:standby 2024-09-19T05:38:46.864 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:38:46.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:46 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3497108373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: Deploying daemon mds.a on smithi038 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6838/2963619728,v1:172.21.15.38:6839/2963619728] up:boot 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: fsmap 10 up:standby 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:38:47.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:46 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3497108373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: Deploying daemon mds.a on smithi038 2024-09-19T05:38:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6838/2963619728,v1:172.21.15.38:6839/2963619728] up:boot 2024-09-19T05:38:47.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: fsmap 10 up:standby 2024-09-19T05:38:47.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:38:47.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:46 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3497108373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:48.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:48 smithi038 ceph-mon[31227]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:48.894 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:48 smithi038 systemd[1]: Starting Ceph mds.a for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:38:49.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:48 smithi179 ceph-mon[35781]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:49.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:48 smithi047 ceph-mon[35327]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:48 smithi038 podman[70352]: 2024-09-19 05:38:48.878883986 +0000 UTC m=+0.016517181 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:48 smithi038 podman[70352]: 2024-09-19 05:38:48.998969389 +0000 UTC m=+0.136602612 container create 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 podman[70352]: 2024-09-19 05:38:49.193114708 +0000 UTC m=+0.330747892 container init 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 podman[70352]: 2024-09-19 05:38:49.196217399 +0000 UTC m=+0.333850585 container start 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 ceph-mds[70371]: -- 172.21.15.38:0/1889659672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4fc91d860 con 0x55a4fc8fa800 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: starting mds.a at 2024-09-19T05:38:49.286 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 bash[70352]: 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a 2024-09-19T05:38:49.644 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:38:49 smithi038 systemd[1]: Started Ceph mds.a for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:49.863 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:50.126 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:50.515 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.515 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: Deploying daemon mds.b on smithi047 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6840/3698066275,v1:172.21.15.38:6841/3698066275] up:boot 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: fsmap 11 up:standby 2024-09-19T05:38:50.516 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:50 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:38:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 -- 172.21.15.38:0/3763219140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 msgr2=0x7fd2701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 --2- 172.21.15.38:0/3763219140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd2701067e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd260009900 tx=0x7fd26002a410 comp rx=0 tx=0).stop 2024-09-19T05:38:50.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 -- 172.21.15.38:0/3763219140 shutdown_connections 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 --2- 172.21.15.38:0/3763219140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270106f10 0x7fd27010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 --2- 172.21.15.38:0/3763219140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd2701067e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 --2- 172.21.15.38:0/3763219140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270101de0 0x7fd2701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.597+0000 7fd277268640 1 -- 172.21.15.38:0/3763219140 >> 172.21.15.38:0/3763219140 conn(0x7fd2700fd8c0 msgr2=0x7fd2700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- 172.21.15.38:0/3763219140 shutdown_connections 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- 172.21.15.38:0/3763219140 wait complete. 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 Processor -- start 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- start start 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 0x7fd270198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270102790 0x7fd270198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:50.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111f70 con 0x7fd270106f10 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111df0 con 0x7fd270102790 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.598+0000 7fd277268640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701120f0 con 0x7fd270101de0 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275805640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275805640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41434/0 (socket says 172.21.15.38:41434) 2024-09-19T05:38:50.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 0x7fd270198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275805640 1 -- 172.21.15.38:0/3705210905 learned_addr learned my addr 172.21.15.38:0/3705210905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd274803640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270102790 0x7fd270198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 -- 172.21.15.38:0/3705210905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270102790 msgr2=0x7fd270198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270102790 0x7fd270198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 -- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 msgr2=0x7fd27019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 -- 172.21.15.38:0/3705210905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27019d400 con 0x7fd270101de0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275805640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd275004640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 0x7fd270198450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00d6d0 tx=0x7fd26c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c017070 con 0x7fd270101de0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27019d6f0 con 0x7fd270101de0 2024-09-19T05:38:50.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c009820 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c004cd0 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.599+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701a4fd0 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.600+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2701021c0 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.602+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd26c0099d0 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.602+0000 7fd2667fc640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 0x7fd25c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.602+0000 7fd274803640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 0x7fd25c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.602+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd26c0ceff0 con 0x7fd270101de0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.603+0000 7fd274803640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 0x7fd25c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd260009230 tx=0x7fd2600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:50.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.605+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd26c09bac0 con 0x7fd270101de0 2024-09-19T05:38:50.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: Deploying daemon mds.b on smithi047 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6840/3698066275,v1:172.21.15.38:6841/3698066275] up:boot 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: fsmap 11 up:standby 2024-09-19T05:38:50.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:50 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: Deploying daemon mds.b on smithi047 2024-09-19T05:38:50.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6840/3698066275,v1:172.21.15.38:6841/3698066275] up:boot 2024-09-19T05:38:50.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: fsmap 11 up:standby 2024-09-19T05:38:50.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:50 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:38:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.816+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd27010a150 con 0x7fd270101de0 2024-09-19T05:38:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.817+0000 7fd2667fc640 1 -- 172.21.15.38:0/3705210905 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7fd26c09fdc0 con 0x7fd270101de0 2024-09-19T05:38:50.819 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:50.819 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":12,"btime":"2024-09-19T05:38:49:748923+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:38:50.819 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 12 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 msgr2=0x7fd25c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 0x7fd25c0a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd260009230 tx=0x7fd2600058d0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 msgr2=0x7fd270198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 0x7fd270198450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00d6d0 tx=0x7fd26c00db90 comp rx=0 tx=0).stop 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 shutdown_connections 2024-09-19T05:38:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd25c0a3240 0x7fd25c0a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd270102790 0x7fd270198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 --2- 172.21.15.38:0/3705210905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd270101de0 0x7fd270198450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 >> 172.21.15.38:0/3705210905 conn(0x7fd2700fd8c0 msgr2=0x7fd2700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 shutdown_connections 2024-09-19T05:38:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:50.819+0000 7fd277268640 1 -- 172.21.15.38:0/3705210905 wait complete. 2024-09-19T05:38:51.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:51 smithi038 ceph-mon[31227]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:51 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3705210905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:51.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:51 smithi047 ceph-mon[35327]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:51.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:51 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3705210905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:51.618 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:51 smithi179 ceph-mon[35781]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:51.618 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:51 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3705210905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:52.226 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 05:38:51 smithi047 systemd[1]: Started Ceph mds.b for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:boot 2024-09-19T05:38:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: fsmap 12 up:standby 2024-09-19T05:38:53.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:52 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:38:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:boot 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: fsmap 12 up:standby 2024-09-19T05:38:53.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:52 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:boot 2024-09-19T05:38:53.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: fsmap 12 up:standby 2024-09-19T05:38:53.227 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:52 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:38:53.494 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:38:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:54.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 -- 172.21.15.38:0/2013885669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4103740 msgr2=0x7f5af4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2013885669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4103740 0x7f5af4107b80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009890 tx=0x7f5adc02a380 comp rx=0 tx=0).stop 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 -- 172.21.15.38:0/2013885669 shutdown_connections 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2013885669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 0x7f5af410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2013885669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4103740 0x7f5af4107b80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2013885669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4102d90 0x7f5af4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 -- 172.21.15.38:0/2013885669 >> 172.21.15.38:0/2013885669 conn(0x7f5af40fc7d0 msgr2=0x7f5af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.305+0000 7f5af8bed640 1 -- 172.21.15.38:0/2013885669 shutdown_connections 2024-09-19T05:38:54.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.306+0000 7f5af8bed640 1 -- 172.21.15.38:0/2013885669 wait complete. 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.306+0000 7f5af8bed640 1 Processor -- start 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.306+0000 7f5af8bed640 1 -- start start 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 0x7f5af4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 0x7f5af419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4112090 con 0x7f5af4103740 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4111f10 con 0x7f5af4102d90 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4112210 con 0x7f5af4108290 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 0x7f5af4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:52556/0 (socket says 172.21.15.38:52556) 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 -- 172.21.15.38:0/2167704071 learned_addr learned my addr 172.21.15.38:0/2167704071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:54.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af2d9d640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 0x7f5af419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 -- 172.21.15.38:0/2167704071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 msgr2=0x7f5af419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 0x7f5af419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 -- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 msgr2=0x7f5af4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 0x7f5af4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 -- 172.21.15.38:0/2167704071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af419d5a0 con 0x7f5af4102d90 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5aebfff640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 0x7f5af4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af259c640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9c0 tx=0x7f5ae400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400cde0 con 0x7f5af4102d90 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae40035f0 con 0x7f5af4102d90 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af419d890 con 0x7f5af4102d90 2024-09-19T05:38:54.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4010690 con 0x7f5af4102d90 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.307+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a5170 con 0x7f5af4102d90 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.308+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af4103170 con 0x7f5af4102d90 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.310+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5ae40037a0 con 0x7f5af4102d90 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.310+0000 7f5aeb7fe640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 0x7f5ae00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.310+0000 7f5aebfff640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 0x7f5ae00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.310+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5ae40cfa90 con 0x7f5af4102d90 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.311+0000 7f5aebfff640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 0x7f5ae00a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5adc005940 tx=0x7f5adc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:54.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.313+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ae409cbb0 con 0x7f5af4102d90 2024-09-19T05:38:54.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.573+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af4199950 con 0x7f5af4102d90 2024-09-19T05:38:54.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.574+0000 7f5aeb7fe640 1 -- 172.21.15.38:0/2167704071 <== mon.2 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f5af4199950 con 0x7f5af4102d90 2024-09-19T05:38:54.576 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:54.577 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":13,"btime":"2024-09-19T05:38:52:353504+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:38:54.577 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-09-19T05:38:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 msgr2=0x7f5ae00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 0x7f5ae00a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5adc005940 tx=0x7f5adc0058d0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 msgr2=0x7f5af41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9c0 tx=0x7f5ae400ee80 comp rx=0 tx=0).stop 2024-09-19T05:38:54.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 shutdown_connections 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ae00a32c0 0x7f5ae00a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5af4108290 0x7f5af419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5af4103740 0x7f5af4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 --2- 172.21.15.38:0/2167704071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4102d90 0x7f5af41985f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 >> 172.21.15.38:0/2167704071 conn(0x7f5af40fc7d0 msgr2=0x7f5af4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 shutdown_connections 2024-09-19T05:38:54.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:54.576+0000 7f5af8bed640 1 -- 172.21.15.38:0/2167704071 wait complete. 2024-09-19T05:38:54.888 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.888 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.889 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:54 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:54 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:54.902 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:54.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:54.903 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:54 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:38:55.269 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T05:38:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:55 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2167704071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:55 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:55 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:55 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2167704071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:55 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:55 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:55.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:55 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2167704071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:38:55.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:55 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:55.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:55 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.067 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 -- 172.21.15.38:0/2890079027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0101420 msgr2=0x7f92d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 --2- 172.21.15.38:0/2890079027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0101420 0x7f92d010d950 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009cd0 tx=0x7f92c802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 -- 172.21.15.38:0/2890079027 shutdown_connections 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 --2- 172.21.15.38:0/2890079027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0101420 0x7f92d010d950 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 --2- 172.21.15.38:0/2890079027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0100a80 0x7f92d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 --2- 172.21.15.38:0/2890079027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 0x7f92d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 -- 172.21.15.38:0/2890079027 >> 172.21.15.38:0/2890079027 conn(0x7f92d00fc7a0 msgr2=0x7f92d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 -- 172.21.15.38:0/2890079027 shutdown_connections 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.541+0000 7f92d4c33640 1 -- 172.21.15.38:0/2890079027 wait complete. 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.542+0000 7f92d4c33640 1 Processor -- start 2024-09-19T05:38:56.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.542+0000 7f92d4c33640 1 -- start start 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0101420 0x7f92d0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 0x7f92d019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111fa0 con 0x7f92d0100a80 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111e20 con 0x7f92d0106a80 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92d4c33640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0112120 con 0x7f92d0101420 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41464/0 (socket says 172.21.15.38:41464) 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 -- 172.21.15.38:0/1690002257 learned_addr learned my addr 172.21.15.38:0/1690002257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92cdd9b640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0101420 0x7f92d0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ced9d640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 0x7f92d019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:56.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 -- 172.21.15.38:0/1690002257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0101420 msgr2=0x7f92d0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0101420 0x7f92d0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 -- 172.21.15.38:0/1690002257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 msgr2=0x7f92d019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 0x7f92d019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 -- 172.21.15.38:0/1690002257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d019d3f0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92ce59c640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d950 tx=0x7f92c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0017070 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.543+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c0009c30 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.544+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c000f6b0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.544+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d019d6e0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.544+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a4fc0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.545+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d0100ee0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.546+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f92c00109d0 con 0x7f92d0100a80 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.547+0000 7f92bf7fe640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 0x7f92b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:56.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.547+0000 7f92cdd9b640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 0x7f92b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.547+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f92c00cf150 con 0x7f92d0100a80 2024-09-19T05:38:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.547+0000 7f92cdd9b640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 0x7f92b80a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92c4005fd0 tx=0x7f92c4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:56.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.550+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92c0009de0 con 0x7f92d0100a80 2024-09-19T05:38:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.739+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f92d01997a0 con 0x7f92d0100a80 2024-09-19T05:38:56.741 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.741 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:56 smithi038 ceph-mon[31227]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:56.741 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.741 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:56 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.740+0000 7f92bf7fe640 1 -- 172.21.15.38:0/1690002257 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f92d01997a0 con 0x7f92d0100a80 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:38:56.742 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-19T05:38:56.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 msgr2=0x7f92b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:56.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 0x7f92b80a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92c4005fd0 tx=0x7f92c4005e20 comp rx=0 tx=0).stop 2024-09-19T05:38:56.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 msgr2=0x7f92d0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:56.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92c000d950 tx=0x7f92c000de10 comp rx=0 tx=0).stop 2024-09-19T05:38:56.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 shutdown_connections 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92b80a3240 0x7f92b80a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92d0106a80 0x7f92d019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92d0101420 0x7f92d0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 --2- 172.21.15.38:0/1690002257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92d0100a80 0x7f92d0198440 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 >> 172.21.15.38:0/1690002257 conn(0x7f92d00fc7a0 msgr2=0x7f92d00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 shutdown_connections 2024-09-19T05:38:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:56.742+0000 7f92d4c33640 1 -- 172.21.15.38:0/1690002257 wait complete. 2024-09-19T05:38:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:56 smithi179 ceph-mon[35781]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:56 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:56 smithi047 ceph-mon[35327]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:56.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:56 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T05:38:57.417 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T05:38:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1690002257' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:38:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:57.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.687 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1690002257' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:38:57.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:57.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:57.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1690002257' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:38:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:38:57.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:38:57.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:38:58.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 -- 172.21.15.38:0/4050689175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc102d40 msgr2=0x7f9ecc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/4050689175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc102d40 0x7f9ecc103120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009900 tx=0x7f9ebc02a480 comp rx=0 tx=0).stop 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 -- 172.21.15.38:0/4050689175 shutdown_connections 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/4050689175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc10c170 0x7f9ecc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/4050689175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc103660 0x7f9ecc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/4050689175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc102d40 0x7f9ecc103120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 -- 172.21.15.38:0/4050689175 >> 172.21.15.38:0/4050689175 conn(0x7f9ecc0fc7a0 msgr2=0x7f9ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.168+0000 7f9ed25e2640 1 -- 172.21.15.38:0/4050689175 shutdown_connections 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 -- 172.21.15.38:0/4050689175 wait complete. 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 Processor -- start 2024-09-19T05:38:58.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 -- start start 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc102d40 0x7f9ecc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 0x7f9ecc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111f70 con 0x7f9ecc10c170 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111df0 con 0x7f9ecc103660 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.169+0000 7f9ed25e2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1120f0 con 0x7f9ecc102d40 2024-09-19T05:38:58.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37620/0 (socket says 172.21.15.38:37620) 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 -- 172.21.15.38:0/2406350922 learned_addr learned my addr 172.21.15.38:0/2406350922 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ecbfff640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc102d40 0x7f9ecc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ecb7fe640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 0x7f9ecc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 -- 172.21.15.38:0/2406350922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc102d40 msgr2=0x7f9ecc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc102d40 0x7f9ecc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 -- 172.21.15.38:0/2406350922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 msgr2=0x7f9ecc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 0x7f9ecc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 -- 172.21.15.38:0/2406350922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc1a69a0 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ecb7fe640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 0x7f9ecc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed0b7f640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eaa0 tx=0x7f9ec400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400cdb0 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4005020 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc1a6c90 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40116b0 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.170+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a7180 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.171+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc103120 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.173+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9ec4005560 con 0x7f9ecc10c170 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.173+0000 7f9ec97fa640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 0x7f9eb40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.173+0000 7f9ecbfff640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 0x7f9eb40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:38:58.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.173+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f9ec40cf680 con 0x7f9ecc10c170 2024-09-19T05:38:58.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.174+0000 7f9ecbfff640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 0x7f9eb40a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009230 tx=0x7f9ebc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:38:58.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.177+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9ec4016030 con 0x7f9ecc10c170 2024-09-19T05:38:58.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.356+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9ecc1071e0 con 0x7f9ecc10c170 2024-09-19T05:38:58.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.838+0000 7f9ec97fa640 1 -- 172.21.15.38:0/2406350922 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f9ec409c840 con 0x7f9ecc10c170 2024-09-19T05:38:58.841 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-19T05:38:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 msgr2=0x7f9eb40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 0x7f9eb40a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009230 tx=0x7f9ebc0058d0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 msgr2=0x7f9ecc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:38:58.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eaa0 tx=0x7f9ec400ef60 comp rx=0 tx=0).stop 2024-09-19T05:38:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 shutdown_connections 2024-09-19T05:38:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9eb40a3240 0x7f9eb40a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc10c170 0x7f9ecc101420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ecc103660 0x7f9ecc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 --2- 172.21.15.38:0/2406350922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ecc102d40 0x7f9ecc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:38:58.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.841+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 >> 172.21.15.38:0/2406350922 conn(0x7f9ecc0fc7a0 msgr2=0x7f9ecc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:38:58.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.842+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 shutdown_connections 2024-09-19T05:38:58.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:38:58.842+0000 7f9ed25e2640 1 -- 172.21.15.38:0/2406350922 wait complete. 2024-09-19T05:38:59.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:58 smithi038 ceph-mon[31227]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:59.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:58 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:38:59.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:58 smithi179 ceph-mon[35781]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:59.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:58 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:38:59.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:58 smithi047 ceph-mon[35327]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:38:59.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:58 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:38:59.510 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-19T05:38:59.768 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:38:59.849 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:59 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:38:59.849 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:38:59 smithi038 ceph-mon[31227]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:39:00.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:59 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:39:00.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:38:59 smithi179 ceph-mon[35781]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:39:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:59 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2406350922' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:39:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:38:59 smithi047 ceph-mon[35327]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 -- 172.21.15.38:0/3775826499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c102d40 msgr2=0x7fa73c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 --2- 172.21.15.38:0/3775826499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c102d40 0x7fa73c103120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009900 tx=0x7fa72c02a480 comp rx=0 tx=0).stop 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 -- 172.21.15.38:0/3775826499 shutdown_connections 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 --2- 172.21.15.38:0/3775826499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c10c170 0x7fa73c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 --2- 172.21.15.38:0/3775826499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c103660 0x7fa73c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 --2- 172.21.15.38:0/3775826499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c102d40 0x7fa73c103120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 -- 172.21.15.38:0/3775826499 >> 172.21.15.38:0/3775826499 conn(0x7fa73c0fc7a0 msgr2=0x7fa73c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 -- 172.21.15.38:0/3775826499 shutdown_connections 2024-09-19T05:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.251+0000 7fa741ce6640 1 -- 172.21.15.38:0/3775826499 wait complete. 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 Processor -- start 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 -- start start 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c102d40 0x7fa73c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c103660 0x7fa73c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c111f70 con 0x7fa73c10c170 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c111df0 con 0x7fa73c102d40 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa741ce6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1120f0 con 0x7fa73c103660 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.252+0000 7fa73bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37636/0 (socket says 172.21.15.38:37636) 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 -- 172.21.15.38:0/2185667556 learned_addr learned my addr 172.21.15.38:0/2185667556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:00.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73b7fe640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c102d40 0x7fa73c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 -- 172.21.15.38:0/2185667556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c103660 msgr2=0x7fa73c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c103660 0x7fa73c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 -- 172.21.15.38:0/2185667556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c102d40 msgr2=0x7fa73c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c102d40 0x7fa73c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 -- 172.21.15.38:0/2185667556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c19d400 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa73bfff640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa73400eaa0 tx=0x7fa73400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73400cdb0 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa734005020 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c19d6f0 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7340116b0 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.253+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c1a4fd0 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.254+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73c103120 con 0x7fa73c10c170 2024-09-19T05:39:00.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.256+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa734005560 con 0x7fa73c10c170 2024-09-19T05:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.256+0000 7fa738ff9640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 0x7fa7240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.256+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7fa7340cfc50 con 0x7fa73c10c170 2024-09-19T05:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.256+0000 7fa73b7fe640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 0x7fa7240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.257+0000 7fa73b7fe640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 0x7fa7240a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0096c0 tx=0x7fa72c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.260+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa734016030 con 0x7fa73c10c170 2024-09-19T05:39:00.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.437+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa73c1997b0 con 0x7fa73c10c170 2024-09-19T05:39:00.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.871+0000 7fa738ff9640 1 -- 172.21.15.38:0/2185667556 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) ==== 171+0+0 (secure 0 0 0) 0x7fa73409cca0 con 0x7fa73c10c170 2024-09-19T05:39:00.873 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-19T05:39:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.873+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 msgr2=0x7fa7240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.873+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 0x7fa7240a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0096c0 tx=0x7fa72c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.873+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 msgr2=0x7fa73c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.873+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa73400eaa0 tx=0x7fa73400ef60 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 shutdown_connections 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa7240a3240 0x7fa7240a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa73c10c170 0x7fa73c19cd20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c103660 0x7fa73c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 --2- 172.21.15.38:0/2185667556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73c102d40 0x7fa73c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 >> 172.21.15.38:0/2185667556 conn(0x7fa73c0fc7a0 msgr2=0x7fa73c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 shutdown_connections 2024-09-19T05:39:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:00.874+0000 7fa741ce6640 1 -- 172.21.15.38:0/2185667556 wait complete. 2024-09-19T05:39:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:00 smithi038 ceph-mon[31227]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:39:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:00 smithi038 ceph-mon[31227]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:00 smithi179 ceph-mon[35781]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:39:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:00 smithi179 ceph-mon[35781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:01.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:00 smithi047 ceph-mon[35327]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:39:01.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:00 smithi047 ceph-mon[35327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:01.560 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T05:39:01.815 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:01 smithi179 ceph-mon[35781]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:01 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T05:39:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:01 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T05:39:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:01 smithi179 ceph-mon[35781]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:39:02.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:01 smithi179 ceph-mon[35781]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:39:02.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:01 smithi047 ceph-mon[35327]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:02.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:01 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T05:39:02.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:01 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T05:39:02.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:01 smithi047 ceph-mon[35327]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:39:02.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:01 smithi047 ceph-mon[35327]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:39:02.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:01 smithi038 ceph-mon[31227]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:02.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:01 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T05:39:02.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:01 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2185667556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T05:39:02.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:01 smithi038 ceph-mon[31227]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:39:02.235 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:01 smithi038 ceph-mon[31227]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:39:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- 172.21.15.38:0/3959238069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4106f10 msgr2=0x7f6da410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 --2- 172.21.15.38:0/3959238069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4106f10 0x7f6da410ea90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009ca0 tx=0x7f6d9c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- 172.21.15.38:0/3959238069 shutdown_connections 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 --2- 172.21.15.38:0/3959238069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4106f10 0x7f6da410ea90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 --2- 172.21.15.38:0/3959238069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4102790 0x7f6da41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 --2- 172.21.15.38:0/3959238069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4101de0 0x7f6da41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- 172.21.15.38:0/3959238069 >> 172.21.15.38:0/3959238069 conn(0x7f6da40fd8c0 msgr2=0x7f6da40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- 172.21.15.38:0/3959238069 shutdown_connections 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- 172.21.15.38:0/3959238069 wait complete. 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 Processor -- start 2024-09-19T05:39:02.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.318+0000 7f6daa52f640 1 -- start start 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4101de0 0x7f6da41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 0x7f6da419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6da4111f70 con 0x7f6da4102790 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6da4111df0 con 0x7f6da4101de0 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6daa52f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6da41120f0 con 0x7f6da4106f10 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37662/0 (socket says 172.21.15.38:37662) 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 -- 172.21.15.38:0/991982079 learned_addr learned my addr 172.21.15.38:0/991982079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da3fff640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4101de0 0x7f6da41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da8acc640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 0x7f6da419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 -- 172.21.15.38:0/991982079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 msgr2=0x7f6da419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 0x7f6da419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 -- 172.21.15.38:0/991982079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4101de0 msgr2=0x7f6da41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4101de0 0x7f6da41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 -- 172.21.15.38:0/991982079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da419d380 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da8acc640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 0x7f6da419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.319+0000 7f6da37fe640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800e9f0 tx=0x7f6d9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.320+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800cd80 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.320+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d98010910 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.320+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da419d670 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.320+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98018990 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.320+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da41a4f50 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.321+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da41021c0 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.322+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6d98010ac0 con 0x7f6da4102790 2024-09-19T05:39:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.323+0000 7f6da17fa640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 0x7f6d8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.323+0000 7f6da3fff640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 0x7f6d8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.323+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f6d980cfea0 con 0x7f6da4102790 2024-09-19T05:39:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.323+0000 7f6da3fff640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 0x7f6d8c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005fd0 tx=0x7f6d9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.326+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6d9809cd70 con 0x7f6da4102790 2024-09-19T05:39:02.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.542+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6da410a1b0 con 0x7f6da4102790 2024-09-19T05:39:02.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.898+0000 7f6da17fa640 1 -- 172.21.15.38:0/991982079 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6d9809c670 con 0x7f6da4102790 2024-09-19T05:39:02.901 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 msgr2=0x7f6d8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 0x7f6d8c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005fd0 tx=0x7f6d9400b390 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 msgr2=0x7f6da4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800e9f0 tx=0x7f6d9800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 shutdown_connections 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d8c0a3240 0x7f6d8c0a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da4106f10 0x7f6da419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da4102790 0x7f6da4198910 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 --2- 172.21.15.38:0/991982079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6da4101de0 0x7f6da41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 >> 172.21.15.38:0/991982079 conn(0x7f6da40fd8c0 msgr2=0x7f6da40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:02.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 shutdown_connections 2024-09-19T05:39:02.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:02.901+0000 7f6daa52f640 1 -- 172.21.15.38:0/991982079 wait complete. 2024-09-19T05:39:03.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:02 smithi179 ceph-mon[35781]: pgmap v275: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:03.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:02 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:39:03.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:02 smithi179 ceph-mon[35781]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T05:39:03.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:02 smithi047 ceph-mon[35327]: pgmap v275: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:03.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:02 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:39:03.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:02 smithi047 ceph-mon[35327]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T05:39:03.281 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:02 smithi038 ceph-mon[31227]: pgmap v275: 97 pgs: 12 creating+peering, 7 active+clean, 78 unknown; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:03.281 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:02 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:39:03.281 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:02 smithi038 ceph-mon[31227]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-19T05:39:03.632 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-19T05:39:03.633 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-19T05:39:03.893 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:04.171 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:03 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:39:04.172 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:03 smithi038 ceph-mon[31227]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:39:04.172 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:03 smithi038 ceph-mon[31227]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T05:39:04.172 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:03 smithi038 ceph-mon[31227]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T05:39:04.197 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: 2024-09-19T05:39:03.999+0000 7f14c5323640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T05:39:04.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:03 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:39:04.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:03 smithi179 ceph-mon[35781]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:39:04.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:03 smithi179 ceph-mon[35781]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T05:39:04.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:03 smithi179 ceph-mon[35781]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T05:39:04.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:03 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/991982079' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:39:04.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:03 smithi047 ceph-mon[35327]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:39:04.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:03 smithi047 ceph-mon[35327]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-19T05:39:04.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:03 smithi047 ceph-mon[35327]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-19T05:39:04.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3347951672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc101df0 msgr2=0x7fd3bc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:04.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3347951672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1021d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009900 tx=0x7fd3b802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:04.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3347951672 shutdown_connections 2024-09-19T05:39:04.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3347951672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc106f20 0x7fd3bc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3347951672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3347951672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1021d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.380+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3347951672 >> 172.21.15.38:0/3347951672 conn(0x7fd3bc0fd8b0 msgr2=0x7fd3bc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3347951672 shutdown_connections 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3347951672 wait complete. 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 Processor -- start 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 -- start start 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc106f20 0x7fd3bc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:04.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.381+0000 7fd3c38d8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc111f80 con 0x7fd3bc1027a0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c38d8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc111e00 con 0x7fd3bc101df0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c38d8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc112100 con 0x7fd3bc106f20 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37680/0 (socket says 172.21.15.38:37680) 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 -- 172.21.15.38:0/3704999964 learned_addr learned my addr 172.21.15.38:0/3704999964 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c1e75640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc106f20 0x7fd3bc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c1674640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 -- 172.21.15.38:0/3704999964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc106f20 msgr2=0x7fd3bc19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc106f20 0x7fd3bc19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 -- 172.21.15.38:0/3704999964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc101df0 msgr2=0x7fd3bc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 -- 172.21.15.38:0/3704999964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc19d490 con 0x7fd3bc1027a0 2024-09-19T05:39:04.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c0e73640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00d950 tx=0x7fd3ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017030 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc19d780 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac004020 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0047a0 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.382+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc1a5060 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.384+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc1021d0 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.385+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd3ac0041d0 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.385+0000 7fd3b27fc640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 0x7fd3a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.386+0000 7fd3c1674640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 0x7fd3a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.386+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd3ac0cf0f0 con 0x7fd3bc1027a0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.386+0000 7fd3c1674640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 0x7fd3a80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009230 tx=0x7fd3b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:04.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.390+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3ac09bf60 con 0x7fd3bc1027a0 2024-09-19T05:39:04.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.574+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fd3bc10a100 con 0x7fd3bc1027a0 2024-09-19T05:39:04.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.920+0000 7fd3b27fc640 1 -- 172.21.15.38:0/3704999964 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) ==== 157+0+0 (secure 0 0 0) 0x7fd3ac09b860 con 0x7fd3bc1027a0 2024-09-19T05:39:04.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 msgr2=0x7fd3a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:04.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 0x7fd3a80a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009230 tx=0x7fd3b80058d0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 msgr2=0x7fd3bc198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00d950 tx=0x7fd3ac00de10 comp rx=0 tx=0).stop 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 shutdown_connections 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd3a80a3240 0x7fd3a80a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3bc106f20 0x7fd3bc19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3bc1027a0 0x7fd3bc198a20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 --2- 172.21.15.38:0/3704999964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3bc101df0 0x7fd3bc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.922+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 >> 172.21.15.38:0/3704999964 conn(0x7fd3bc0fd8b0 msgr2=0x7fd3bc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.923+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 shutdown_connections 2024-09-19T05:39:04.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:04.923+0000 7fd3c38d8640 1 -- 172.21.15.38:0/3704999964 wait complete. 2024-09-19T05:39:05.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: pgmap v278: 97 pgs: 19 creating+peering, 56 active+clean, 22 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T05:39:05.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:39:05.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: Cluster is now healthy 2024-09-19T05:39:05.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:active 2024-09-19T05:39:05.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:04 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: pgmap v278: 97 pgs: 19 creating+peering, 56 active+clean, 22 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: Cluster is now healthy 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:active 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:04 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: pgmap v278: 97 pgs: 19 creating+peering, 56 active+clean, 22 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: Cluster is now healthy 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] up:active 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:05.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:04 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T05:39:05.635 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T05:39:05.898 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:06.183 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:05 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T05:39:06.183 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:05 smithi038 ceph-mon[31227]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:39:06.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:05 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T05:39:06.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:05 smithi179 ceph-mon[35781]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:39:06.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:05 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3704999964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T05:39:06.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:05 smithi047 ceph-mon[35327]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 -- 172.21.15.38:0/3380013277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 msgr2=0x7f23b010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3380013277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b010bb50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2398009900 tx=0x7f239802a480 comp rx=0 tx=0).stop 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 -- 172.21.15.38:0/3380013277 shutdown_connections 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3380013277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3380013277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b010bb50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:06.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3380013277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 0x7f23b0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 -- 172.21.15.38:0/3380013277 >> 172.21.15.38:0/3380013277 conn(0x7f23b00fc7a0 msgr2=0x7f23b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 -- 172.21.15.38:0/3380013277 shutdown_connections 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.384+0000 7f23b50fb640 1 -- 172.21.15.38:0/3380013277 wait complete. 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 Processor -- start 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 -- start start 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 0x7f23b01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:06.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0111f70 con 0x7f23b0103660 2024-09-19T05:39:06.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0111df0 con 0x7f23b010c170 2024-09-19T05:39:06.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23b50fb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01120f0 con 0x7f23b0102d40 2024-09-19T05:39:06.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:06.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37692/0 (socket says 172.21.15.38:37692) 2024-09-19T05:39:06.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.385+0000 7f23ae59c640 1 -- 172.21.15.38:0/3987866958 learned_addr learned my addr 172.21.15.38:0/3987866958 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:06.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23aed9d640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 0x7f23b01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23af59e640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 -- 172.21.15.38:0/3987866958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 msgr2=0x7f23b01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 0x7f23b01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 -- 172.21.15.38:0/3987866958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 msgr2=0x7f23b019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 -- 172.21.15.38:0/3987866958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b019d360 con 0x7f23b0103660 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23af59e640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23ae59c640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2398005ab0 tx=0x7f23980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239803f030 con 0x7f23b0103660 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2398003ce0 con 0x7f23b0103660 2024-09-19T05:39:06.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b019d5f0 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398035070 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.386+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b01a4f30 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.387+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b006b070 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.388+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2398042030 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.390+0000 7f23a7fff640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 0x7f239c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.390+0000 7f23aed9d640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 0x7f239c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.390+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8277+0+0 (secure 0 0 0) 0x7f23980f0160 con 0x7f23b0103660 2024-09-19T05:39:06.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.390+0000 7f23aed9d640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 0x7f239c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23a800abe0 tx=0x7f23a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:06.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.394+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23980bce50 con 0x7f23b0103660 2024-09-19T05:39:06.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:06.573+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f23b01071e0 con 0x7f23b0103660 2024-09-19T05:39:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:07 smithi038 ceph-mon[31227]: pgmap v280: 97 pgs: 13 creating+peering, 84 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-19T05:39:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:07 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:39:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:07 smithi179 ceph-mon[35781]: pgmap v280: 97 pgs: 13 creating+peering, 84 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-19T05:39:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:07 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:39:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:07 smithi047 ceph-mon[35327]: pgmap v280: 97 pgs: 13 creating+peering, 84 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-19T05:39:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:07 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:39:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.930+0000 7f23a7fff640 1 -- 172.21.15.38:0/3987866958 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) ==== 228+0+0 (secure 0 0 0) 0x7f23980bc750 con 0x7f23b0103660 2024-09-19T05:39:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' created 2024-09-19T05:39:07.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.933+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 msgr2=0x7f239c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:07.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.933+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 0x7f239c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23a800abe0 tx=0x7f23a800a2d0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.933+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 msgr2=0x7f23b01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:07.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.933+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2398005ab0 tx=0x7f23980042f0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 shutdown_connections 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f239c0a3240 0x7f239c0a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23b010c170 0x7f23b019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23b0103660 0x7f23b01988f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 --2- 172.21.15.38:0/3987866958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102d40 0x7f23b01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 >> 172.21.15.38:0/3987866958 conn(0x7f23b00fc7a0 msgr2=0x7f23b00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 shutdown_connections 2024-09-19T05:39:07.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:07.934+0000 7f23b50fb640 1 -- 172.21.15.38:0/3987866958 wait complete. 2024-09-19T05:39:08.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:08 smithi038 ceph-mon[31227]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:39:08.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:08 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:39:08.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:08 smithi038 ceph-mon[31227]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:39:08.620 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-19T05:39:08.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:08 smithi179 ceph-mon[35781]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:39:08.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:08 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:39:08.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:08 smithi179 ceph-mon[35781]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:39:08.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:08 smithi047 ceph-mon[35327]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:39:08.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:08 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3987866958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:39:08.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:08 smithi047 ceph-mon[35327]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:39:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 -- 172.21.15.38:0/1107555310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494100e20 msgr2=0x7ff494101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 --2- 172.21.15.38:0/1107555310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494100e20 0x7ff494101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff484009900 tx=0x7ff48402a480 comp rx=0 tx=0).stop 2024-09-19T05:39:09.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 -- 172.21.15.38:0/1107555310 shutdown_connections 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 --2- 172.21.15.38:0/1107555310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102ef0 0x7ff49410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 --2- 172.21.15.38:0/1107555310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494102570 0x7ff4941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 --2- 172.21.15.38:0/1107555310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494100e20 0x7ff494101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 -- 172.21.15.38:0/1107555310 >> 172.21.15.38:0/1107555310 conn(0x7ff4940fc7d0 msgr2=0x7ff4940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 -- 172.21.15.38:0/1107555310 shutdown_connections 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.349+0000 7ff499b93640 1 -- 172.21.15.38:0/1107555310 wait complete. 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 Processor -- start 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 -- start start 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 0x7ff4940fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102570 0x7ff4940fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:09.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff494110f80 con 0x7ff494102ef0 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff494110e00 con 0x7ff494100e20 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff499b93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff494111100 con 0x7ff494102570 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff493fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff493fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38250/0 (socket says 172.21.15.38:38250) 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff493fff640 1 -- 172.21.15.38:0/227127720 learned_addr learned my addr 172.21.15.38:0/227127720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff4937fe640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 0x7ff4940fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.350+0000 7ff48affd640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102570 0x7ff4940fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 -- 172.21.15.38:0/227127720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102570 msgr2=0x7ff4940fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102570 0x7ff4940fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 -- 172.21.15.38:0/227127720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 msgr2=0x7ff4940fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 0x7ff4940fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 -- 172.21.15.38:0/227127720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941a6930 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff4937fe640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 0x7ff4940fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff493fff640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00cad0 tx=0x7ff48c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00ae20 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941a6c20 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff48c005020 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00f6b0 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.351+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941a7160 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.352+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff494101200 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.354+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff48c0051d0 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.355+0000 7ff4917fa640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 0x7ff4800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.355+0000 7ff4937fe640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 0x7ff4800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.355+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7ff48c0cfef0 con 0x7ff494102ef0 2024-09-19T05:39:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.356+0000 7ff4937fe640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 0x7ff4800a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:09.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.359+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff48c014030 con 0x7ff494102ef0 2024-09-19T05:39:09.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.539+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff4940fe1b0 con 0x7ff494102ef0 2024-09-19T05:39:09.542 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:09 smithi038 ceph-mon[31227]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-19T05:39:09.542 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:09 smithi038 ceph-mon[31227]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:39:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:09 smithi179 ceph-mon[35781]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-19T05:39:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:09 smithi179 ceph-mon[35781]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:39:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:09 smithi047 ceph-mon[35327]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-19T05:39:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:09 smithi047 ceph-mon[35327]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:39:09.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.938+0000 7ff4917fa640 1 -- 172.21.15.38:0/227127720 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7ff48c09c990 con 0x7ff494102ef0 2024-09-19T05:39:09.940 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 allow_ec_overwrites to true 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 msgr2=0x7ff4800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 0x7ff4800a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484005a20 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 msgr2=0x7ff4940fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00cad0 tx=0x7ff48c00cf90 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 shutdown_connections 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4800a3240 0x7ff4800a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494102ef0 0x7ff4940fd700 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff494102570 0x7ff4940fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 --2- 172.21.15.38:0/227127720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff494100e20 0x7ff4940fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 >> 172.21.15.38:0/227127720 conn(0x7ff4940fc7d0 msgr2=0x7ff49406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:09.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 shutdown_connections 2024-09-19T05:39:09.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:09.942+0000 7ff499b93640 1 -- 172.21.15.38:0/227127720 wait complete. 2024-09-19T05:39:10.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T05:39:10.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T05:39:10.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:10 smithi038 ceph-mon[31227]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:39:10.670 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-19T05:39:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T05:39:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T05:39:10.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:10 smithi179 ceph-mon[35781]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:39:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T05:39:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/227127720' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T05:39:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:10 smithi047 ceph-mon[35327]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:39:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- 172.21.15.38:0/1045696510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04681068d0 msgr2=0x7f0468106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 --2- 172.21.15.38:0/1045696510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04681068d0 0x7f0468106cb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0464009930 tx=0x7f046402a420 comp rx=0 tx=0).stop 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- 172.21.15.38:0/1045696510 shutdown_connections 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 --2- 172.21.15.38:0/1045696510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0468101270 0x7f046810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 --2- 172.21.15.38:0/1045696510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681008d0 0x7f0468100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 --2- 172.21.15.38:0/1045696510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04681068d0 0x7f0468106cb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- 172.21.15.38:0/1045696510 >> 172.21.15.38:0/1045696510 conn(0x7f04680fc610 msgr2=0x7f04680fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- 172.21.15.38:0/1045696510 shutdown_connections 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- 172.21.15.38:0/1045696510 wait complete. 2024-09-19T05:39:11.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 Processor -- start 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.449+0000 7f046efa2640 1 -- start start 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04681008d0 0x7f0468197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681068d0 0x7f046819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110cd0 con 0x7f0468101270 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110b50 con 0x7f04681068d0 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0468110e50 con 0x7f04681008d0 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38262/0 (socket says 172.21.15.38:38262) 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 -- 172.21.15.38:0/1767679385 learned_addr learned my addr 172.21.15.38:0/1767679385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046dfa0640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04681008d0 0x7f0468197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:11.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046e7a1640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681068d0 0x7f046819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 -- 172.21.15.38:0/1767679385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04681008d0 msgr2=0x7f0468197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04681008d0 0x7f0468197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 -- 172.21.15.38:0/1767679385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681068d0 msgr2=0x7f046819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681068d0 0x7f046819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 -- 172.21.15.38:0/1767679385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046819c0b0 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046d79f640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f045800c920 tx=0x7f045800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458007c70 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046819c3a0 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0458007e10 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.450+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458018890 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.451+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a3c80 con 0x7f0468101270 2024-09-19T05:39:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.451+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0468100d30 con 0x7f0468101270 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.453+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0458018a70 con 0x7f0468101270 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.453+0000 7f045effd640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 0x7f04540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.453+0000 7f046dfa0640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 0x7f04540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.453+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f04580cfc80 con 0x7f0468101270 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.454+0000 7f046dfa0640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 0x7f04540a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0464009900 tx=0x7f04640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:11.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.457+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0458014030 con 0x7f0468101270 2024-09-19T05:39:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:11 smithi038 ceph-mon[31227]: pgmap v285: 129 pgs: 13 creating+peering, 17 unknown, 99 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-19T05:39:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:11 smithi038 ceph-mon[31227]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:11.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:11.676+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f04681a3fb0 con 0x7f0468101270 2024-09-19T05:39:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:11 smithi179 ceph-mon[35781]: pgmap v285: 129 pgs: 13 creating+peering, 17 unknown, 99 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-19T05:39:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:11 smithi179 ceph-mon[35781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:11.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:11 smithi047 ceph-mon[35327]: pgmap v285: 129 pgs: 13 creating+peering, 17 unknown, 99 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-19T05:39:11.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:11 smithi047 ceph-mon[35327]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:39:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.403+0000 7f045effd640 1 -- 172.21.15.38:0/1767679385 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f045809c780 con 0x7f0468101270 2024-09-19T05:39:12.405 INFO:teuthology.orchestra.run.smithi038.stderr:added data pool 4 to fsmap 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 msgr2=0x7f04540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 0x7f04540a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0464009900 tx=0x7f04640058d0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 msgr2=0x7f0468197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f045800c920 tx=0x7f045800cde0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 shutdown_connections 2024-09-19T05:39:12.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f04540a3240 0x7f04540a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04681068d0 0x7f046819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0468101270 0x7f0468197640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 --2- 172.21.15.38:0/1767679385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04681008d0 0x7f0468197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.406+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 >> 172.21.15.38:0/1767679385 conn(0x7f04680fc610 msgr2=0x7f04680fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.407+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 shutdown_connections 2024-09-19T05:39:12.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:12.407+0000 7f046efa2640 1 -- 172.21.15.38:0/1767679385 wait complete. 2024-09-19T05:39:12.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:12 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T05:39:12.730 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:12 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T05:39:12.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:12 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T05:39:13.129 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:39:13.395 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:13.648 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:13 smithi038 ceph-mon[31227]: pgmap v287: 129 pgs: 13 creating+peering, 9 unknown, 107 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 440 B/s wr, 1 op/s 2024-09-19T05:39:13.648 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:13 smithi038 ceph-mon[31227]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:39:13.648 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:13 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T05:39:13.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:13 smithi038 ceph-mon[31227]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:13 smithi179 ceph-mon[35781]: pgmap v287: 129 pgs: 13 creating+peering, 9 unknown, 107 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 440 B/s wr, 1 op/s 2024-09-19T05:39:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:13 smithi179 ceph-mon[35781]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:39:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:13 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T05:39:13.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:13 smithi179 ceph-mon[35781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:13 smithi047 ceph-mon[35327]: pgmap v287: 129 pgs: 13 creating+peering, 9 unknown, 107 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 440 B/s wr, 1 op/s 2024-09-19T05:39:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:13 smithi047 ceph-mon[35327]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:39:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:13 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1767679385' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T05:39:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:13 smithi047 ceph-mon[35327]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:13.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 -- 172.21.15.38:0/3578900788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498106f10 msgr2=0x7f949810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 --2- 172.21.15.38:0/3578900788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498106f10 0x7f949810ea90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9490009ca0 tx=0x7f949002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 -- 172.21.15.38:0/3578900788 shutdown_connections 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 --2- 172.21.15.38:0/3578900788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498106f10 0x7f949810ea90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 --2- 172.21.15.38:0/3578900788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498102790 0x7f94981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 --2- 172.21.15.38:0/3578900788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f94981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 -- 172.21.15.38:0/3578900788 >> 172.21.15.38:0/3578900788 conn(0x7f94980fd8c0 msgr2=0x7f94980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 -- 172.21.15.38:0/3578900788 shutdown_connections 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.859+0000 7f949da72640 1 -- 172.21.15.38:0/3578900788 wait complete. 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 Processor -- start 2024-09-19T05:39:13.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 -- start start 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f9498198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498106f10 0x7f949819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9498111f90 con 0x7f9498102790 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9498111e10 con 0x7f9498106f10 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.860+0000 7f949da72640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9498112110 con 0x7f9498101de0 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38276/0 (socket says 172.21.15.38:38276) 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 -- 172.21.15.38:0/3676212110 learned_addr learned my addr 172.21.15.38:0/3676212110 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9497fff640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498106f10 0x7f949819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:13.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f94977fe640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f9498198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 -- 172.21.15.38:0/3676212110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 msgr2=0x7f9498198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f9498198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 -- 172.21.15.38:0/3676212110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498106f10 msgr2=0x7f949819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498106f10 0x7f949819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 -- 172.21.15.38:0/3676212110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949819d420 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f94977fe640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f9498198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9496ffd640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f948c00d950 tx=0x7f948c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c017070 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949819d710 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c009c30 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c00f6b0 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.861+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981a4ff0 con 0x7f9498102790 2024-09-19T05:39:13.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.862+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94981021c0 con 0x7f9498102790 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.863+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f948c0109d0 con 0x7f9498102790 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.864+0000 7f9494ff9640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 0x7f94800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.864+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f948c0cf7c0 con 0x7f9498102790 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.864+0000 7f94977fe640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 0x7f94800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.864+0000 7f94977fe640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 0x7f94800a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9488009930 tx=0x7f9488008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:13.868+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f948c009de0 con 0x7f9498102790 2024-09-19T05:39:14.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.078+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f949810a150 con 0x7f9498102790 2024-09-19T05:39:14.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.079+0000 7f9494ff9640 1 -- 172.21.15.38:0/3676212110 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f948c09c2a0 con 0x7f9498102790 2024-09-19T05:39:14.082 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:14.082 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-19T05:39:12:393730+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:11.677668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24496,"qdb_cluster":[24496]},"id":1}]} 2024-09-19T05:39:14.082 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 msgr2=0x7f94800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 0x7f94800a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9488009930 tx=0x7f9488008040 comp rx=0 tx=0).stop 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 msgr2=0x7f94981989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f948c00d950 tx=0x7f948c00de10 comp rx=0 tx=0).stop 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 shutdown_connections 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94800a3240 0x7f94800a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:14.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498106f10 0x7f949819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:14.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9498102790 0x7f94981989b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:14.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 --2- 172.21.15.38:0/3676212110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9498101de0 0x7f9498198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:14.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 >> 172.21.15.38:0/3676212110 conn(0x7f94980fd8c0 msgr2=0x7f94980fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:14.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 shutdown_connections 2024-09-19T05:39:14.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:14.081+0000 7f949da72640 1 -- 172.21.15.38:0/3676212110 wait complete. 2024-09-19T05:39:14.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:14 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3676212110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:14.722 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:14 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3676212110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:14.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:14 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3676212110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:14.753 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/3202291860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 msgr2=0x7fa1a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/3202291860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a41021c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c02a410 comp rx=0 tx=0).stop 2024-09-19T05:39:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/3202291860 shutdown_connections 2024-09-19T05:39:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/3202291860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4106f10 0x7fa1a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/3202291860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4102790 0x7fa1a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/3202291860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a41021c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/3202291860 >> 172.21.15.38:0/3202291860 conn(0x7fa1a40fd8c0 msgr2=0x7fa1a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/3202291860 shutdown_connections 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.506+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/3202291860 wait complete. 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 Processor -- start 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 -- start start 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4102790 0x7fa1a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 0x7fa1a419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111f70 con 0x7fa1a4101de0 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111df0 con 0x7fa1a4106f10 2024-09-19T05:39:15.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1ac1f3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41120f0 con 0x7fa1a4102790 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38296/0 (socket says 172.21.15.38:38296) 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 -- 172.21.15.38:0/796751372 learned_addr learned my addr 172.21.15.38:0/796751372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a978e640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4102790 0x7fa1a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1aa790640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 0x7fa1a419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 -- 172.21.15.38:0/796751372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4102790 msgr2=0x7fa1a4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4102790 0x7fa1a4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 -- 172.21.15.38:0/796751372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 msgr2=0x7fa1a419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 0x7fa1a419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1a9f8f640 1 -- 172.21.15.38:0/796751372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419d3e0 con 0x7fa1a4101de0 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.507+0000 7fa1aa790640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 0x7fa1a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1a9f8f640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa19c006010 tx=0x7fa19c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:15.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c03f030 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419d670 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c003ce0 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c035070 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.508+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a4fb0 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.509+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a41021c0 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.510+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa19c042030 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.511+0000 7fa1a2ffd640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 0x7fa18c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.511+0000 7fa1a978e640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 0x7fa18c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.511+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fa19c0f0110 con 0x7fa1a4101de0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.512+0000 7fa1a978e640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 0x7fa18c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa198005fd0 tx=0x7fa198009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.516+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa19c0bccb0 con 0x7fa1a4101de0 2024-09-19T05:39:15.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.693+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1a4199790 con 0x7fa1a4101de0 2024-09-19T05:39:15.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:15 smithi038 ceph-mon[31227]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:15.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:15 smithi038 ceph-mon[31227]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T05:39:15.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:15 smithi038 ceph-mon[31227]: Cluster is now healthy 2024-09-19T05:39:15.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.695+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/796751372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7fa19c0bc5b0 con 0x7fa1a4101de0 2024-09-19T05:39:15.697 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:15.697 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:39:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:15 smithi179 ceph-mon[35781]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:15 smithi179 ceph-mon[35781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T05:39:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:15 smithi179 ceph-mon[35781]: Cluster is now healthy 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 msgr2=0x7fa18c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 0x7fa18c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa198005fd0 tx=0x7fa198009470 comp rx=0 tx=0).stop 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 msgr2=0x7fa1a4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa19c006010 tx=0x7fa19c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 shutdown_connections 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa18c0a3240 0x7fa18c0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1a4106f10 0x7fa1a419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1a4102790 0x7fa1a4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 --2- 172.21.15.38:0/796751372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a4101de0 0x7fa1a4198430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:15.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 >> 172.21.15.38:0/796751372 conn(0x7fa1a40fd8c0 msgr2=0x7fa1a40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:15.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 shutdown_connections 2024-09-19T05:39:15.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:15.697+0000 7fa1ac1f3640 1 -- 172.21.15.38:0/796751372 wait complete. 2024-09-19T05:39:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:15 smithi047 ceph-mon[35327]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:15 smithi047 ceph-mon[35327]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T05:39:15.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:15 smithi047 ceph-mon[35327]: Cluster is now healthy 2024-09-19T05:39:16.422 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:16 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/796751372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:16.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:16.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:16 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/796751372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:16.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:16 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/796751372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:17.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 -- 172.21.15.38:0/3566547446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 msgr2=0x7f0b6810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 --2- 172.21.15.38:0/3566547446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b6810bb50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c02a480 comp rx=0 tx=0).stop 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 -- 172.21.15.38:0/3566547446 shutdown_connections 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 --2- 172.21.15.38:0/3566547446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b6810c170 0x7f0b6810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 --2- 172.21.15.38:0/3566547446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b6810bb50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 --2- 172.21.15.38:0/3566547446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b68102d40 0x7f0b68103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.164+0000 7f0b70250640 1 -- 172.21.15.38:0/3566547446 >> 172.21.15.38:0/3566547446 conn(0x7f0b680fc7a0 msgr2=0x7f0b680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- 172.21.15.38:0/3566547446 shutdown_connections 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- 172.21.15.38:0/3566547446 wait complete. 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 Processor -- start 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- start start 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b68102d40 0x7f0b68198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b6810c170 0x7f0b6819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68111f70 con 0x7f0b68103660 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68111df0 con 0x7f0b6810c170 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b70250640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681120f0 con 0x7f0b68102d40 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b6d7eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.165+0000 7f0b6d7eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38300/0 (socket says 172.21.15.38:38300) 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 -- 172.21.15.38:0/3949895263 learned_addr learned my addr 172.21.15.38:0/3949895263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6dfec640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b68102d40 0x7f0b68198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6e7ed640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b6810c170 0x7f0b6819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 -- 172.21.15.38:0/3949895263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b68102d40 msgr2=0x7f0b68198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b68102d40 0x7f0b68198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 -- 172.21.15.38:0/3949895263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b6810c170 msgr2=0x7f0b6819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b6810c170 0x7f0b6819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 -- 172.21.15.38:0/3949895263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6819d420 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b6d7eb640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006010 tx=0x7f0b5c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c03f030 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6819d6b0 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b5c004440 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c035070 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.166+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a4ff0 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.167+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b68103120 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.168+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0b5c042030 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.169+0000 7f0b66ffd640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 0x7f0b500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.169+0000 7f0b6dfec640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 0x7f0b500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.169+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f0b5c0f0010 con 0x7f0b68103660 2024-09-19T05:39:17.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.170+0000 7f0b6dfec640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 0x7f0b500a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b60005fd0 tx=0x7f0b60009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:17.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.173+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b5c0bcaf0 con 0x7f0b68103660 2024-09-19T05:39:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.351+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b68106c50 con 0x7f0b68103660 2024-09-19T05:39:17.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.353+0000 7f0b66ffd640 1 -- 172.21.15.38:0/3949895263 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7f0b5c0bc3f0 con 0x7f0b68103660 2024-09-19T05:39:17.355 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:17.355 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:39:17.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 msgr2=0x7f0b500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 0x7f0b500a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b60005fd0 tx=0x7f0b60009500 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 msgr2=0x7f0b681989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006010 tx=0x7f0b5c004040 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 shutdown_connections 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0b500a3240 0x7f0b500a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b6810c170 0x7f0b6819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b68103660 0x7f0b681989b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 --2- 172.21.15.38:0/3949895263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b68102d40 0x7f0b68198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 >> 172.21.15.38:0/3949895263 conn(0x7f0b680fc7a0 msgr2=0x7f0b680fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 shutdown_connections 2024-09-19T05:39:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:17.355+0000 7f0b70250640 1 -- 172.21.15.38:0/3949895263 wait complete. 2024-09-19T05:39:17.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:17 smithi038 ceph-mon[31227]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:17.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3949895263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:17 smithi179 ceph-mon[35781]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:17 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3949895263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:17.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:17 smithi047 ceph-mon[35327]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:17 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3949895263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:18.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:39:18.297 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 -- 172.21.15.38:0/261528377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 msgr2=0x7ff4dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/261528377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc10ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009ca0 tx=0x7ff4d402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 -- 172.21.15.38:0/261528377 shutdown_connections 2024-09-19T05:39:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/261528377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc10ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/261528377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc102790 0x7ff4dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/261528377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc101de0 0x7ff4dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 -- 172.21.15.38:0/261528377 >> 172.21.15.38:0/261528377 conn(0x7ff4dc0fd8c0 msgr2=0x7ff4dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 -- 172.21.15.38:0/261528377 shutdown_connections 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.765+0000 7ff4e21ed640 1 -- 172.21.15.38:0/261528377 wait complete. 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 Processor -- start 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 -- start start 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc101de0 0x7ff4dc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc102790 0x7ff4dc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc111fc0 con 0x7ff4dc106f10 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc111e40 con 0x7ff4dc102790 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.766+0000 7ff4e21ed640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112140 con 0x7ff4dc101de0 2024-09-19T05:39:18.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42550/0 (socket says 172.21.15.38:42550) 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 -- 172.21.15.38:0/1692652003 learned_addr learned my addr 172.21.15.38:0/1692652003 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4db7fe640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc101de0 0x7ff4dc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4daffd640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc102790 0x7ff4dc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 -- 172.21.15.38:0/1692652003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc101de0 msgr2=0x7ff4dc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc101de0 0x7ff4dc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 -- 172.21.15.38:0/1692652003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc102790 msgr2=0x7ff4dc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc102790 0x7ff4dc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 -- 172.21.15.38:0/1692652003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc19d350 con 0x7ff4dc106f10 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4dbfff640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4033040 tx=0x7ff4d4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4031650 con 0x7ff4dc106f10 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d40317f0 con 0x7ff4dc106f10 2024-09-19T05:39:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc19d5e0 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d402fa40 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.767+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a4f20 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.768+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4dc1021c0 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.769+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff4d402fc60 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.770+0000 7ff4d8ff9640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 0x7ff4c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.770+0000 7ff4db7fe640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 0x7ff4c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.770+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7ff4d40f10f0 con 0x7ff4dc106f10 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.771+0000 7ff4db7fe640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 0x7ff4c40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009b60 tx=0x7ff4cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.774+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4d40bdbd0 con 0x7ff4dc106f10 2024-09-19T05:39:18.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.985+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4dc10a0f0 con 0x7ff4dc106f10 2024-09-19T05:39:18.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.985+0000 7ff4d8ff9640 1 -- 172.21.15.38:0/1692652003 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7ff4d4039e10 con 0x7ff4dc106f10 2024-09-19T05:39:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:18.988 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-19T05:39:12:393730+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:11.677668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24496,"qdb_cluster":[24496]},"id":1}]} 2024-09-19T05:39:18.988 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.987+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 msgr2=0x7ff4c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.987+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 0x7ff4c40a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009b60 tx=0x7ff4cc005f90 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 msgr2=0x7ff4dc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4033040 tx=0x7ff4d4003f10 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 shutdown_connections 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff4c40a3240 0x7ff4c40a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4dc106f10 0x7ff4dc19cc70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4dc102790 0x7ff4dc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 --2- 172.21.15.38:0/1692652003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4dc101de0 0x7ff4dc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 >> 172.21.15.38:0/1692652003 conn(0x7ff4dc0fd8c0 msgr2=0x7ff4dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:18.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 shutdown_connections 2024-09-19T05:39:18.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:18.988+0000 7ff4e21ed640 1 -- 172.21.15.38:0/1692652003 wait complete. 2024-09-19T05:39:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:19 smithi179 ceph-mon[35781]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:19 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1692652003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:19 smithi047 ceph-mon[35327]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:19 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1692652003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:19.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:19 smithi038 ceph-mon[31227]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:19.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:19 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1692652003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:19.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:20.057 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:20.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 -- 172.21.15.38:0/3963563524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c101de0 msgr2=0x7f373c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:20.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 --2- 172.21.15.38:0/3963563524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c101de0 0x7f373c1021c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3738009900 tx=0x7f373802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:20.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 -- 172.21.15.38:0/3963563524 shutdown_connections 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 --2- 172.21.15.38:0/3963563524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 --2- 172.21.15.38:0/3963563524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c102790 0x7f373c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 --2- 172.21.15.38:0/3963563524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c101de0 0x7f373c1021c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 -- 172.21.15.38:0/3963563524 >> 172.21.15.38:0/3963563524 conn(0x7f373c0fd8c0 msgr2=0x7f373c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 -- 172.21.15.38:0/3963563524 shutdown_connections 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.524+0000 7f3743a63640 1 -- 172.21.15.38:0/3963563524 wait complete. 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 Processor -- start 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 -- start start 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c101de0 0x7f373c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f373c111f30 con 0x7f373c102790 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f373c111db0 con 0x7f373c101de0 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.525+0000 7f3743a63640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f373c1120b0 con 0x7f373c106f10 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42556/0 (socket says 172.21.15.38:42556) 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 -- 172.21.15.38:0/998249786 learned_addr learned my addr 172.21.15.38:0/998249786 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f37417ff640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c101de0 0x7f373c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3742000640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 -- 172.21.15.38:0/998249786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 msgr2=0x7f373c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 -- 172.21.15.38:0/998249786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c101de0 msgr2=0x7f373c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c101de0 0x7f373c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 -- 172.21.15.38:0/998249786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c19d4c0 con 0x7f373c102790 2024-09-19T05:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3742000640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3740ffe640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f372c00d950 tx=0x7f372c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c017030 con 0x7f373c102790 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c004020 con 0x7f373c102790 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c19d7b0 con 0x7f373c102790 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c0047a0 con 0x7f373c102790 2024-09-19T05:39:20.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.526+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c1a5090 con 0x7f373c102790 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.527+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373c1021c0 con 0x7f373c102790 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.529+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f372c0041d0 con 0x7f373c102790 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.529+0000 7f37327fc640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 0x7f37280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.530+0000 7f37417ff640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 0x7f37280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.530+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f372c0cfbd0 con 0x7f373c102790 2024-09-19T05:39:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.530+0000 7f37417ff640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 0x7f37280a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3738009230 tx=0x7f37380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:20.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.533+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f372c09c6b0 con 0x7f373c102790 2024-09-19T05:39:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.712+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f373c10a120 con 0x7f373c102790 2024-09-19T05:39:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.714+0000 7f37327fc640 1 -- 172.21.15.38:0/998249786 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7f372c09bfb0 con 0x7f373c102790 2024-09-19T05:39:20.717 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:20.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:39:20.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 msgr2=0x7f37280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 0x7f37280a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3738009230 tx=0x7f37380058d0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 msgr2=0x7f373c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f372c00d950 tx=0x7f372c00de10 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 shutdown_connections 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37280a3240 0x7f37280a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c106f10 0x7f373c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f373c102790 0x7f373c198a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 --2- 172.21.15.38:0/998249786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f373c101de0 0x7f373c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 >> 172.21.15.38:0/998249786 conn(0x7f373c0fd8c0 msgr2=0x7f373c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 shutdown_connections 2024-09-19T05:39:20.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:20.717+0000 7f3743a63640 1 -- 172.21.15.38:0/998249786 wait complete. 2024-09-19T05:39:21.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:21 smithi038 ceph-mon[31227]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:21.551 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:21 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/998249786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:21.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T05:39:02.544314+0000', 'modified': '2024-09-19T05:39:11.677668+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.179:6839/2081935059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2081935059}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2081935059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24496, 'qdb_cluster': [24496]} 2024-09-19T05:39:21.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T05:39:21.586 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:39:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:21 smithi179 ceph-mon[35781]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:21 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/998249786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:21 smithi047 ceph-mon[35327]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:21 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/998249786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:21.848 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 -- 172.21.15.38:0/774235140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 msgr2=0x7f0184100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 --2- 172.21.15.38:0/774235140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f0184100ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0174009930 tx=0x7f017402a420 comp rx=0 tx=0).stop 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 -- 172.21.15.38:0/774235140 shutdown_connections 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 --2- 172.21.15.38:0/774235140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f018410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 --2- 172.21.15.38:0/774235140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f0184100ee0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 --2- 172.21.15.38:0/774235140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 0x7f0184106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 -- 172.21.15.38:0/774235140 >> 172.21.15.38:0/774235140 conn(0x7f01840fc7a0 msgr2=0x7f01840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 -- 172.21.15.38:0/774235140 shutdown_connections 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.326+0000 7f018bc8d640 1 -- 172.21.15.38:0/774235140 wait complete. 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 Processor -- start 2024-09-19T05:39:22.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 -- start start 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f0184199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 0x7f01841a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0184110e80 con 0x7f0184100a80 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0184110d00 con 0x7f0184101420 2024-09-19T05:39:22.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f018bc8d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0184111000 con 0x7f0184106a80 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42562/0 (socket says 172.21.15.38:42562) 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f0189a29640 1 -- 172.21.15.38:0/1891470800 learned_addr learned my addr 172.21.15.38:0/1891470800 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.327+0000 7f0189228640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f0184199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f018a22a640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 0x7f01841a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 -- 172.21.15.38:0/1891470800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 msgr2=0x7f01841a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 0x7f01841a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 -- 172.21.15.38:0/1891470800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 msgr2=0x7f0184199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f0184199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 -- 172.21.15.38:0/1891470800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841a48b0 con 0x7f0184100a80 2024-09-19T05:39:22.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189228640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f0184199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f0189a29640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f018000e9f0 tx=0x7f018000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f017affd640 1 -- 172.21.15.38:0/1891470800 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018000cd80 con 0x7f0184100a80 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841a4a80 con 0x7f0184100a80 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f017affd640 1 -- 172.21.15.38:0/1891470800 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0180005500 con 0x7f0184100a80 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f017affd640 1 -- 172.21.15.38:0/1891470800 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180010690 con 0x7f0184100a80 2024-09-19T05:39:22.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.328+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a4f40 con 0x7f0184100a80 2024-09-19T05:39:22.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f017affd640 1 -- 172.21.15.38:0/1891470800 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f01800046c0 con 0x7f0184100a80 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f018e066740 1 Processor -- start 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f018e066740 1 -- start start 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f018e066740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f018e066740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.331+0000 7f018e066740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f018e066740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561576157090 con 0x5615761f12f0 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f018e066740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5615760fea70 con 0x5615761f09d0 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f018e066740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561576153050 con 0x5615761fbbb0 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42576/0 (socket says 172.21.15.38:42576) 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 -- 172.21.15.38:0/3129265987 learned_addr learned my addr 172.21.15.38:0/3129265987 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189228640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f018a22a640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f017affd640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 0x7f01700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f018a22a640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 0x7f01700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f017affd640 1 -- 172.21.15.38:0/1891470800 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f01800cffe0 con 0x7f0184100a80 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 -- 172.21.15.38:0/3129265987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 msgr2=0x5615761fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761fdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 -- 172.21.15.38:0/3129265987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 msgr2=0x5615761f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 -- 172.21.15.38:0/3129265987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157610b130 con 0x5615761f12f0 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189228640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:39:22.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.332+0000 7f0189a29640 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f018009c080 tx=0x7f0180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d5b98239b8e16a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f016ffff640 1 -- 172.21.15.38:0/3129265987 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018009bec0 con 0x5615761f12f0 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f016ffff640 1 -- 172.21.15.38:0/3129265987 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018009ca80 con 0x5615761f12f0 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018a22a640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 0x7f01700a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f017c009fd0 tx=0x7f017c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f016ffff640 1 -- 172.21.15.38:0/3129265987 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018009bce0 con 0x5615761f12f0 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 -- 172.21.15.38:0/3129265987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 msgr2=0x5615761fb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f018009c080 tx=0x7f0180004a40 comp rx=0 tx=0).stop 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 -- 172.21.15.38:0/3129265987 shutdown_connections 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x5615761fb670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 --2- 172.21.15.38:0/3129265987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.333+0000 7f018e066740 1 -- 172.21.15.38:0/3129265987 >> 172.21.15.38:0/3129265987 conn(0x5615761ecf60 msgr2=0x5615761ed340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- 172.21.15.38:0/3129265987 shutdown_connections 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- 172.21.15.38:0/3129265987 wait complete. 2024-09-19T05:39:22.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- start start 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 Processor -- start 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761fb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761f12f0 0x5615761f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5615761fe720 con 0x5615761fbbb0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561576111f20 con 0x5615761f09d0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5615761fe8a0 con 0x5615761f12f0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018e066740 10 client.0 _ll_register_callbacks cb 0x5615760db030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.334+0000 7f018a22a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42588/0 (socket says 172.21.15.38:42588) 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f0189a29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761fb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 -- 172.21.15.38:0/159510141 learned_addr learned my addr 172.21.15.38:0/159510141 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f0189228640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761f12f0 0x5615761f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 -- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761f12f0 msgr2=0x5615761f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761f12f0 0x5615761f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 -- 172.21.15.38:0/159510141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 msgr2=0x5615761fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761fb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 -- 172.21.15.38:0/159510141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561575ec82b0 con 0x5615761fbbb0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018a22a640 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f017c00ba30 tx=0x7f017c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b3b2f04285e8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f016d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c027060 con 0x5615761fbbb0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f017c0045d0 con 0x5615761fbbb0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018009ca80 con 0x5615761fbbb0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f017c02e050 con 0x5615761fbbb0 2024-09-19T05:39:22.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018e066740 10 client.14715 fetch_fsmap learned FSMap version 17 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.335+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56157620a0f0 con 0x5615761fbbb0 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018009ca80 con 0x5615761fbbb0 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f017c010c40 con 0x5615761fbbb0 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 10 client.14715 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 20 client.14715 populate_metadata read hostname 'smithi038' 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157620c740 0x56157620dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 4 client.14715 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561576164600 con 0x56157620c740 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.336+0000 7f0189a29640 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157620c740 0x56157620dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f016d7fa640 10 client.14715 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f0189a29640 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157620c740 0x56157620dbb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7749ebb14f24c03 server_cookie=cd1b241bad55e553 in_seq=0 out_seq=0 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f016d7fa640 1 -- 172.21.15.38:0/159510141 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561576164600 con 0x56157620c740 2024-09-19T05:39:22.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f016d7fa640 10 client.14715 handle_command_reply: tid=0 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 client.14715 shutdown 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 msgr2=0x5615761f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f017c00ba30 tx=0x7f017c004a40 comp rx=0 tx=0).stop 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 shutdown_connections 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157620c740 0x56157620dbb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761fbbb0 0x5615761f8d20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761f12f0 0x5615761f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 --2- 172.21.15.38:0/159510141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615761fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 >> 172.21.15.38:0/159510141 conn(0x5615761ecf60 msgr2=0x5615761f2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.337+0000 7f016d7fa640 0 client.14715 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.338+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 shutdown_connections 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.338+0000 7f018e066740 1 -- 172.21.15.38:0/159510141 wait complete. 2024-09-19T05:39:22.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.338+0000 7f018e066740 20 client.14715 trim_cache size 0 max 16384 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 Processor -- start 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 -- start start 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 0x5615761f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56157620bef0 con 0x5615761f09d0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56157620c5c0 con 0x5615761f12f0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018e066740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56157620cc90 con 0x5615761fbbb0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42600/0 (socket says 172.21.15.38:42600) 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 -- 172.21.15.38:0/1419080807 learned_addr learned my addr 172.21.15.38:0/1419080807 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f018a22a640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189228640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 0x5615761f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 -- 172.21.15.38:0/1419080807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 msgr2=0x5615761f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 -- 172.21.15.38:0/1419080807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 msgr2=0x5615761f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 0x5615761f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189a29640 1 -- 172.21.15.38:0/1419080807 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615761ecd90 con 0x5615761f09d0 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.356+0000 7f0189228640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 0x5615761f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f0189a29640 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f018009c310 tx=0x7f018009bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36f259c9e62a992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f016f7fe640 1 -- 172.21.15.38:0/1419080807 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018009ca80 con 0x5615761f09d0 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f016f7fe640 1 -- 172.21.15.38:0/1419080807 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01800d37f0 con 0x5615761f09d0 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f016f7fe640 1 -- 172.21.15.38:0/1419080807 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018009c5e0 con 0x5615761f09d0 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 -- 172.21.15.38:0/1419080807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 msgr2=0x56157620ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f018009c310 tx=0x7f018009bd00 comp rx=0 tx=0).stop 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 -- 172.21.15.38:0/1419080807 shutdown_connections 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x5615761f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f12f0 0x5615761f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 --2- 172.21.15.38:0/1419080807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f09d0 0x56157620ce10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 -- 172.21.15.38:0/1419080807 >> 172.21.15.38:0/1419080807 conn(0x5615761ecf60 msgr2=0x5615761f7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 -- 172.21.15.38:0/1419080807 shutdown_connections 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.357+0000 7f018e066740 1 -- 172.21.15.38:0/1419080807 wait complete. 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 -- start start 2024-09-19T05:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 Processor -- start 2024-09-19T05:39:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615762167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x56157620f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56157620da30 con 0x5615761f12f0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56157620d360 con 0x5615761f09d0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56157620e100 con 0x5615761fbbb0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018e066740 10 client.0 _ll_register_callbacks cb 0x7f017c009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f0189228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f0189228640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42604/0 (socket says 172.21.15.38:42604) 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f0189228640 1 -- 172.21.15.38:0/3947009549 learned_addr learned my addr 172.21.15.38:0/3947009549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f018a22a640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x56157620f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.358+0000 7f0189a29640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615762167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 -- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 msgr2=0x56157620f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x56157620f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 -- 172.21.15.38:0/3947009549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 msgr2=0x5615762167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615762167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 -- 172.21.15.38:0/3947009549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157620f9a0 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189a29640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615762167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f018a22a640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x56157620f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f0189228640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f017402a940 tx=0x7f01740319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a089c110e6e3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f016d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017404a020 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f0174004250 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01800d37f0 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0174052050 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f018e066740 10 client.14721 fetch_fsmap learned FSMap version 17 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x561576275f10 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.359+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01800d37f0 con 0x5615761f12f0 2024-09-19T05:39:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f0174045070 con 0x5615761f12f0 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 10 client.14721 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 20 client.14721 populate_metadata read hostname 'smithi038' 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157627ab70 0x561576279880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 4 client.14721 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "status"}) -- 0x56157615eb10 con 0x56157627ab70 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f0189a29640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157627ab70 0x561576279880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f0189a29640 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157627ab70 0x561576279880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b81cdb71ff00b5 server_cookie=767da2688a6e2d7e in_seq=0 out_seq=0 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f016d7fa640 10 client.14721 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f016d7fa640 1 -- 172.21.15.38:0/3947009549 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56157615eb10 con 0x56157627ab70 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f016d7fa640 10 client.14721 handle_command_reply: tid=0 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.360+0000 7f018e066740 1 client.14721 shutdown 2024-09-19T05:39:22.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 msgr2=0x56157620eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f017402a940 tx=0x7f01740319b0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 shutdown_connections 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56157627ab70 0x561576279880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615761fbbb0 0x56157620f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5615761f12f0 0x56157620eb70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 --2- 172.21.15.38:0/3947009549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615761f09d0 0x5615762167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f016d7fa640 0 client.14721 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 >> 172.21.15.38:0/3947009549 conn(0x5615761ecf60 msgr2=0x5615761f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 shutdown_connections 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 1 -- 172.21.15.38:0/3947009549 wait complete. 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.361+0000 7f018e066740 20 client.14721 trim_cache size 0 max 16384 2024-09-19T05:39:22.370 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:39:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24496, 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 19.461036964000002, 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 17, 2024-09-19T05:39:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 87, 2024-09-19T05:39:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:39:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 39.319792145999998 2024-09-19T05:39:22.374 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 msgr2=0x7f01700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 0x7f01700a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f017c009fd0 tx=0x7f017c004ab0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 msgr2=0x7f01841992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f018000e9f0 tx=0x7f018000edf0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 shutdown_connections 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f01700a3240 0x7f01700a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184106a80 0x7f01841a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0184101420 0x7f0184199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 --2- 172.21.15.38:0/1891470800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0184100a80 0x7f01841992f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:22.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 >> 172.21.15.38:0/1891470800 conn(0x7f01840fc7a0 msgr2=0x7f01840fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:22.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 shutdown_connections 2024-09-19T05:39:22.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:22.362+0000 7f018bc8d640 1 -- 172.21.15.38:0/1891470800 wait complete. 2024-09-19T05:39:23.235 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T05:39:23.251 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:23.251 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T05:39:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:23.633 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:23 smithi038 ceph-mon[31227]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:23 smithi179 ceph-mon[35781]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:23 smithi047 ceph-mon[35327]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:24.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 -- 172.21.15.38:0/241395327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 msgr2=0x7fa6bc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:24.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/241395327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc103110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009930 tx=0x7fa6ac02a420 comp rx=0 tx=0).stop 2024-09-19T05:39:24.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 -- 172.21.15.38:0/241395327 shutdown_connections 2024-09-19T05:39:24.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/241395327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1082a0 0x7fa6bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/241395327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1036e0 0x7fa6bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/241395327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc103110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 -- 172.21.15.38:0/241395327 >> 172.21.15.38:0/241395327 conn(0x7fa6bc0fc7d0 msgr2=0x7fa6bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 -- 172.21.15.38:0/241395327 shutdown_connections 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.009+0000 7fa6c0bab640 1 -- 172.21.15.38:0/241395327 wait complete. 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 Processor -- start 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 -- start start 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1036e0 0x7fa6bc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1082a0 0x7fa6bc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc1120a0 con 0x7fa6bc102d30 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc111f20 con 0x7fa6bc1036e0 2024-09-19T05:39:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6c0bab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc112220 con 0x7fa6bc1082a0 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42634/0 (socket says 172.21.15.38:42634) 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 -- 172.21.15.38:0/4203119395 learned_addr learned my addr 172.21.15.38:0/4203119395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6bad9d640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1082a0 0x7fa6bc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6b9d9b640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1036e0 0x7fa6bc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 -- 172.21.15.38:0/4203119395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1082a0 msgr2=0x7fa6bc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1082a0 0x7fa6bc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 -- 172.21.15.38:0/4203119395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1036e0 msgr2=0x7fa6bc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1036e0 0x7fa6bc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.010+0000 7fa6ba59c640 1 -- 172.21.15.38:0/4203119395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc19d550 con 0x7fa6bc102d30 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6ba59c640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009230 tx=0x7fa6ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac03f030 con 0x7fa6bc102d30 2024-09-19T05:39:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ac003ad0 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc19d7e0 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac035070 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.011+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc1a5120 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.012+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc103110 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.013+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa6ac042030 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.014+0000 7fa6ab7fe640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 0x7fa6a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.014+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fa6ac0f00c0 con 0x7fa6bc102d30 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.014+0000 7fa6b9d9b640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 0x7fa6a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.014+0000 7fa6b9d9b640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 0x7fa6a40a5780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0fee30 tx=0x7fa6b000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:24.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.018+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa6ac0bcb40 con 0x7fa6bc102d30 2024-09-19T05:39:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.234+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa6bc10a2b0 con 0x7fa6bc102d30 2024-09-19T05:39:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.234+0000 7fa6ab7fe640 1 -- 172.21.15.38:0/4203119395 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fa6ac0bc440 con 0x7fa6bc102d30 2024-09-19T05:39:24.237 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T05:39:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 msgr2=0x7fa6a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 0x7fa6a40a5780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0fee30 tx=0x7fa6b000a2d0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 msgr2=0x7fa6bc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009230 tx=0x7fa6ac004270 comp rx=0 tx=0).stop 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 shutdown_connections 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6a40a32c0 0x7fa6a40a5780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6bc1082a0 0x7fa6bc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc1036e0 0x7fa6bc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 --2- 172.21.15.38:0/4203119395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6bc102d30 0x7fa6bc1985a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.236+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 >> 172.21.15.38:0/4203119395 conn(0x7fa6bc0fc7d0 msgr2=0x7fa6bc105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.237+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 shutdown_connections 2024-09-19T05:39:24.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:24.237+0000 7fa6c0bab640 1 -- 172.21.15.38:0/4203119395 wait complete. 2024-09-19T05:39:25.056 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T05:39:25.057 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T05:39:25.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:25.057 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:39:25.077 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42269sec preferred_lft 42269sec 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T05:39:25.078 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-19T05:39:25.079 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T05:39:25.079 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:39:25.079 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:39:25.079 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-19T05:39:25.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-19T05:39:25.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-19T05:39:25.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T05:39:25.080 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:39:25.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:39:25.270 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:25 smithi038 ceph-mon[31227]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:25.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:25 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4203119395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:39:25.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:25.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:25.293 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T05:39:25.318 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-19T05:39:25.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:25.320 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-19T05:39:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-19T05:39:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T05:39:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T05:39:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T05:39:25.391 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T05:39:25.392 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:39:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:25 smithi179 ceph-mon[35781]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:25 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/4203119395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:39:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:25 smithi047 ceph-mon[35327]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:25 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/4203119395' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:39:25.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:39:25.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:25.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:25.674 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-19T05:39:25.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:25.690 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-19T05:39:25.747 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:39:25.747 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:39:25.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:39:25.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-19T05:39:25.747 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:39:25.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:39:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:25.866 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-19T05:39:25.867 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:39:25.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:39:25.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:25.996 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:39:25.996 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:39:25.996 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-19T05:39:25.996 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T05:39:25.996 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:39:26.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:39:26.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:26.074 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-19T05:39:26.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:39:26.074 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:26.090 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T05:39:26.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:39:26.091 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:26.147 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-19T05:39:26.214 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T05:39:26.271 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-19T05:39:26.271 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:39:26.272 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/7fde1d64c7e4556456743bca2de1e5f55a7b40645ef78fea8add9a72910f6f7c/merged 2024-09-19T05:39:26.273 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/7ba22d345ee837948c18e224aaf4e056f642a0beba1e6dc5791b31c306dd2e79/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/0 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/b7f33302f0e00118ed6523b6e2828017b55298c43417e8f912a95678436da0ba/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/92c9c7a2cde02d1a1c22571b7640e0e1342b6be1b83d5cbd21e334d563cd134c/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/c6fe06ac60091b7c9150fb7b626671a6132b77b82be765aa075e7c9d794900b8/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/ab451104bf540a58fdd530fe98745fd85a1a25448808becd8855714344e095fb/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/d6542e6efdba9e96c9dc96a64e26949aaca528fbbb6ccffa6b265fbf9a30a4e5/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/32d67184af8a3a1fd507616d1fc0881ed4f5012903086783a294853efc016611/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/2d9b1dd3256a7af7951cbc26c39b526d88b1d162b5f0715871c5eb50c9af283e/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/c749be004c1625ef9304526faa727ecc6b6a76c5c5193df9ed05bb1358517050/merged 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:39:26.274 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:39:26.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:26.275 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-19T05:39:26.327 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T05:39:26.327 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-19T05:39:26.370 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-19T05:39:26.395 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T05:39:26.452 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-19T05:39:26.452 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:39:26.453 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/7fde1d64c7e4556456743bca2de1e5f55a7b40645ef78fea8add9a72910f6f7c/merged 2024-09-19T05:39:26.454 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/7ba22d345ee837948c18e224aaf4e056f642a0beba1e6dc5791b31c306dd2e79/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/0 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/b7f33302f0e00118ed6523b6e2828017b55298c43417e8f912a95678436da0ba/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/92c9c7a2cde02d1a1c22571b7640e0e1342b6be1b83d5cbd21e334d563cd134c/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/c6fe06ac60091b7c9150fb7b626671a6132b77b82be765aa075e7c9d794900b8/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/ab451104bf540a58fdd530fe98745fd85a1a25448808becd8855714344e095fb/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/d6542e6efdba9e96c9dc96a64e26949aaca528fbbb6ccffa6b265fbf9a30a4e5/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/32d67184af8a3a1fd507616d1fc0881ed4f5012903086783a294853efc016611/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/2d9b1dd3256a7af7951cbc26c39b526d88b1d162b5f0715871c5eb50c9af283e/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/c749be004c1625ef9304526faa727ecc6b6a76c5c5193df9ed05bb1358517050/merged 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:39:26.455 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:39:26.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:26.456 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-19T05:39:26.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:2024-09-19T05:39:26.470+0000 7f462d617480 -1 init, newargv = 0x55c18d994a20 newargc=15 2024-09-19T05:39:26.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[73551]: starting ceph client 2024-09-19T05:39:26.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[73551]: starting fuse 2024-09-19T05:39:26.489 INFO:teuthology.orchestra.run.smithi038.stdout:120 2024-09-19T05:39:26.490 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-19T05:39:26.490 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-09-19T05:39:26.491 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-09-19T05:39:26.492 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-09-19T05:39:26.492 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T05:39:26.492 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-19T05:39:26.492 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.admin")) 2024-09-19T05:39:26.492 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-19T05:39:26.610 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.admin.73551.asok 2024-09-19T05:39:26.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:26.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73551.asok 2024-09-19T05:39:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:26.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73551.asok status 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "admin", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "73551", 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-09-19T05:39:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 14733, 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 14733 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4218749791 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:39:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4218749791 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.14733 192.168.0.1:0/4218749791", 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/4218749791", 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 17, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 87, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-09-19T05:39:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-09-19T05:39:26.761 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:39:26.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:26.766 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-09-19T05:39:26.767 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:26.783 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-09-19T05:39:26.783 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:26.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:26.783 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:26.853 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-19T05:39:26.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:39:26.971 INFO:teuthology.orchestra.run.smithi038.stdout:# file: . 2024-09-19T05:39:26.971 INFO:teuthology.orchestra.run.smithi038.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-19T05:39:26.971 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:26.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:39:26.977 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:39:26.977 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:39:26.996 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:39:27.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:27 smithi038 ceph-mon[31227]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:27.256 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:27 smithi179 ceph-mon[35781]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:27.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:27 smithi047 ceph-mon[35327]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 -- 172.21.15.38:0/2053695082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 msgr2=0x7f94a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 --2- 172.21.15.38:0/2053695082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41067e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9498009900 tx=0x7f949802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 -- 172.21.15.38:0/2053695082 shutdown_connections 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 --2- 172.21.15.38:0/2053695082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4106f10 0x7f94a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 --2- 172.21.15.38:0/2053695082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41067e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 --2- 172.21.15.38:0/2053695082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4101de0 0x7f94a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 -- 172.21.15.38:0/2053695082 >> 172.21.15.38:0/2053695082 conn(0x7f94a40fd8c0 msgr2=0x7f94a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 -- 172.21.15.38:0/2053695082 shutdown_connections 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.771+0000 7f94a99b2640 1 -- 172.21.15.38:0/2053695082 wait complete. 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 Processor -- start 2024-09-19T05:39:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 -- start start 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4101de0 0x7f94a41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4106f10 0x7f94a419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4111fe0 con 0x7f94a4102790 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4111e60 con 0x7f94a4106f10 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a99b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4112160 con 0x7f94a4101de0 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46944/0 (socket says 172.21.15.38:46944) 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a27fc640 1 -- 172.21.15.38:0/3942195713 learned_addr learned my addr 172.21.15.38:0/3942195713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a2ffd640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4101de0 0x7f94a41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.772+0000 7f94a37fe640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4106f10 0x7f94a419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 -- 172.21.15.38:0/3942195713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4101de0 msgr2=0x7f94a41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4101de0 0x7f94a41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 -- 172.21.15.38:0/3942195713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4106f10 msgr2=0x7f94a419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4106f10 0x7f94a419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 -- 172.21.15.38:0/3942195713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a419d450 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a27fc640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9498004770 tx=0x7f9498003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949803f070 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9498031ca0 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a419d6e0 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498035030 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.773+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41a5020 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.774+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a41032c0 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.775+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9498042070 con 0x7f94a4102790 2024-09-19T05:39:27.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.776+0000 7f94a89b0640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 0x7f948c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.777+0000 7f94a2ffd640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 0x7f948c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.777+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f94980f07c0 con 0x7f94a4102790 2024-09-19T05:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.777+0000 7f94a2ffd640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 0x7f948c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f949400bbe0 tx=0x7f949400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.780+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f94980bcbf0 con 0x7f94a4102790 2024-09-19T05:39:27.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.967+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94a410a150 con 0x7f94a4102790 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.967+0000 7f94a89b0640 1 -- 172.21.15.38:0/3942195713 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+592 (secure 0 0 0) 0x7f94980c0ef0 con 0x7f94a4102790 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2329208823 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1466442042 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1122120616 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2032678289 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2581276558 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:27.971 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:27.971 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2319061439 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:27.971 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:27.971 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 msgr2=0x7f948c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 0x7f948c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f949400bbe0 tx=0x7f949400b390 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 msgr2=0x7f94a41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9498004770 tx=0x7f9498003c10 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 shutdown_connections 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f948c0a3240 0x7f948c0a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a4106f10 0x7f94a419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a4102790 0x7f94a41989e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 --2- 172.21.15.38:0/3942195713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94a4101de0 0x7f94a41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 >> 172.21.15.38:0/3942195713 conn(0x7f94a40fd8c0 msgr2=0x7f94a40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 shutdown_connections 2024-09-19T05:39:27.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:27.970+0000 7f94a99b2640 1 -- 172.21.15.38:0/3942195713 wait complete. 2024-09-19T05:39:28.392 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:28 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3942195713' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:28.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:28 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3942195713' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:28.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:28 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3942195713' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:28.723 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:39:28.724 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:39:28.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:39:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:29.352 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:29 smithi038 ceph-mon[31227]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:29.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:29 smithi179 ceph-mon[35781]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:29.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:29 smithi047 ceph-mon[35327]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 -- 172.21.15.38:0/2580206413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4102790 msgr2=0x7fb1f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/2580206413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4102790 0x7fb1f41067e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009900 tx=0x7fb1e802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 -- 172.21.15.38:0/2580206413 shutdown_connections 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/2580206413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4106f10 0x7fb1f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/2580206413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4102790 0x7fb1f41067e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/2580206413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4101de0 0x7fb1f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 -- 172.21.15.38:0/2580206413 >> 172.21.15.38:0/2580206413 conn(0x7fb1f40fd8c0 msgr2=0x7fb1f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 -- 172.21.15.38:0/2580206413 shutdown_connections 2024-09-19T05:39:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.533+0000 7fb1f9b47640 1 -- 172.21.15.38:0/2580206413 wait complete. 2024-09-19T05:39:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 Processor -- start 2024-09-19T05:39:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 -- start start 2024-09-19T05:39:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 0x7fb1f419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4102790 0x7fb1f419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 0x7fb1f419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4111fc0 con 0x7fb1f4106f10 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4111e40 con 0x7fb1f4101de0 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.534+0000 7fb1f9b47640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4112140 con 0x7fb1f4102790 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 0x7fb1f419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 0x7fb1f419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4102790 0x7fb1f419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 0x7fb1f419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:59288/0 (socket says 172.21.15.38:59288) 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f37fe640 1 -- 172.21.15.38:0/1627234267 learned_addr learned my addr 172.21.15.38:0/1627234267 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 -- 172.21.15.38:0/1627234267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4102790 msgr2=0x7fb1f419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4102790 0x7fb1f419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 -- 172.21.15.38:0/1627234267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 msgr2=0x7fb1f419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 0x7fb1f419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 -- 172.21.15.38:0/1627234267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f419f610 con 0x7fb1f4106f10 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f3fff640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 0x7fb1f419ef30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e940 tx=0x7fb1ec00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec004d20 con 0x7fb1f4106f10 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ec005020 con 0x7fb1f4106f10 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f419f900 con 0x7fb1f4106f10 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0116b0 con 0x7fb1f4106f10 2024-09-19T05:39:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.535+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a71e0 con 0x7fb1f4106f10 2024-09-19T05:39:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.536+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f41021c0 con 0x7fb1f4106f10 2024-09-19T05:39:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.538+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb1ec0054d0 con 0x7fb1f4106f10 2024-09-19T05:39:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.538+0000 7fb1f0ff9640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 0x7fb1dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.539+0000 7fb1f37fe640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 0x7fb1dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.539+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb1ec0cfe90 con 0x7fb1f4106f10 2024-09-19T05:39:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.539+0000 7fb1f37fe640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 0x7fb1dc0a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400bca0 tx=0x7fb1e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:29.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.542+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1ec016030 con 0x7fb1f4106f10 2024-09-19T05:39:29.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.731+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1f41a7440 con 0x7fb1f4106f10 2024-09-19T05:39:29.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.731+0000 7fb1f0ff9640 1 -- 172.21.15.38:0/1627234267 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+592 (secure 0 0 0) 0x7fb1ec09c3a0 con 0x7fb1f4106f10 2024-09-19T05:39:29.734 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:29.734 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2329208823 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1466442042 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1122120616 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2032678289 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2581276558 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2319061439 2024-09-20T05:32:09.496697+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:39:29.735 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-19T05:39:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 msgr2=0x7fb1dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 0x7fb1dc0a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400bca0 tx=0x7fb1e400b410 comp rx=0 tx=0).stop 2024-09-19T05:39:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 msgr2=0x7fb1f419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 0x7fb1f419ef30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e940 tx=0x7fb1ec00ee00 comp rx=0 tx=0).stop 2024-09-19T05:39:29.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 shutdown_connections 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb1dc0a32c0 0x7fb1dc0a5780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1f4106f10 0x7fb1f419ef30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1f4102790 0x7fb1f419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 --2- 172.21.15.38:0/1627234267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f4101de0 0x7fb1f419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 >> 172.21.15.38:0/1627234267 conn(0x7fb1f40fd8c0 msgr2=0x7fb1f40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:29.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 shutdown_connections 2024-09-19T05:39:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:29.734+0000 7fb1f9b47640 1 -- 172.21.15.38:0/1627234267 wait complete. 2024-09-19T05:39:30.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:30 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1627234267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:30.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:30 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1627234267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:30.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:30 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1627234267' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:39:30.528 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-09-19T05:39:30.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:30.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-19T05:39:30.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[73551]: fuse finished with error 0 and tester_r 0 2024-09-19T05:39:30.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:39:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:31 smithi038 ceph-mon[31227]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:31 smithi179 ceph-mon[35781]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:31 smithi047 ceph-mon[35327]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-19T05:39:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:33 smithi038 ceph-mon[31227]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:33 smithi179 ceph-mon[35781]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:33.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:33 smithi047 ceph-mon[35327]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:35 smithi038 ceph-mon[31227]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:35 smithi179 ceph-mon[35781]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:35.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:35 smithi047 ceph-mon[35327]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:36.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:39:36.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:39:36.659 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-19T05:39:36.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:36.997 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:37 smithi038 ceph-mon[31227]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:37 smithi179 ceph-mon[35781]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:37.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:37 smithi047 ceph-mon[35327]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 -- 172.21.15.38:0/297733393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 msgr2=0x7f4d1410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/297733393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1410ea80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ca0 tx=0x7f4d0c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 -- 172.21.15.38:0/297733393 shutdown_connections 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/297733393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1410ea80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/297733393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 0x7f4d141067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/297733393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 0x7f4d141021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 -- 172.21.15.38:0/297733393 >> 172.21.15.38:0/297733393 conn(0x7f4d140fd890 msgr2=0x7f4d140ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 -- 172.21.15.38:0/297733393 shutdown_connections 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.520+0000 7f4d1a37b640 1 -- 172.21.15.38:0/297733393 wait complete. 2024-09-19T05:39:37.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 Processor -- start 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 -- start start 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 0x7f4d14198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 0x7f4d141989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14111f60 con 0x7f4d14106f00 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14111de0 con 0x7f4d14101dd0 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d1a37b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141120e0 con 0x7f4d14102780 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d18918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d18918640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35324/0 (socket says 172.21.15.38:35324) 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d18918640 1 -- 172.21.15.38:0/3898156050 learned_addr learned my addr 172.21.15.38:0/3898156050 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.521+0000 7f4d0bfff640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 0x7f4d141989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:37.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d13fff640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 0x7f4d14198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 -- 172.21.15.38:0/3898156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 msgr2=0x7f4d141989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 0x7f4d141989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 -- 172.21.15.38:0/3898156050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 msgr2=0x7f4d14198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 0x7f4d14198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 -- 172.21.15.38:0/3898156050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1419d430 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d18918640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c031a20 tx=0x7f4d0c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c003c30 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1419d6c0 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c003dd0 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c02fcf0 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.522+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d141a5000 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.523+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d141021b0 con 0x7f4d14106f00 2024-09-19T05:39:37.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.524+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4d0c004050 con 0x7f4d14106f00 2024-09-19T05:39:37.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.525+0000 7f4d11ffb640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:37.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.525+0000 7f4d13fff640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:37.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.525+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f4d0c0f0df0 con 0x7f4d14106f00 2024-09-19T05:39:37.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.526+0000 7f4d13fff640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 0x7f4d000a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d04005d00 tx=0x7f4d0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:37.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.529+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4d0c0bd960 con 0x7f4d14106f00 2024-09-19T05:39:37.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.707+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d14109b50 con 0x7f4d14106f00 2024-09-19T05:39:37.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.708+0000 7f4d11ffb640 1 -- 172.21.15.38:0/3898156050 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7f4d0c02fe90 con 0x7f4d14106f00 2024-09-19T05:39:37.710 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:37.710 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:39:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.711+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 msgr2=0x7f4d000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.711+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 0x7f4d000a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d04005d00 tx=0x7f4d0400b410 comp rx=0 tx=0).stop 2024-09-19T05:39:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.711+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 msgr2=0x7f4d1419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.711+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c031a20 tx=0x7f4d0c031a50 comp rx=0 tx=0).stop 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 shutdown_connections 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d14106f00 0x7f4d1419cd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d14102780 0x7f4d141989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 --2- 172.21.15.38:0/3898156050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d14101dd0 0x7f4d14198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 >> 172.21.15.38:0/3898156050 conn(0x7f4d140fd890 msgr2=0x7f4d140fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 shutdown_connections 2024-09-19T05:39:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:37.712+0000 7f4d1a37b640 1 -- 172.21.15.38:0/3898156050 wait complete. 2024-09-19T05:39:38.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:38 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3898156050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:38.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:38 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3898156050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:38.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:38.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:38 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3898156050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:38.731 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.204+0000 7f13d04c1640 1 -- 172.21.15.38:0/3484552738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8106a80 msgr2=0x7f13c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.204+0000 7f13d04c1640 1 --2- 172.21.15.38:0/3484552738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8106a80 0x7f13c8106e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009930 tx=0x7f13b802a420 comp rx=0 tx=0).stop 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 -- 172.21.15.38:0/3484552738 shutdown_connections 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 --2- 172.21.15.38:0/3484552738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8101420 0x7f13c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 --2- 172.21.15.38:0/3484552738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8100a80 0x7f13c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 --2- 172.21.15.38:0/3484552738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8106a80 0x7f13c8106e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 -- 172.21.15.38:0/3484552738 >> 172.21.15.38:0/3484552738 conn(0x7f13c80fc7a0 msgr2=0x7f13c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 -- 172.21.15.38:0/3484552738 shutdown_connections 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 -- 172.21.15.38:0/3484552738 wait complete. 2024-09-19T05:39:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 Processor -- start 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 -- start start 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.205+0000 7f13d04c1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8101420 0x7f13c8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13d04c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8106a80 0x7f13c81a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13d04c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110e80 con 0x7f13c8100a80 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13d04c1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110d00 con 0x7f13c8101420 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13d04c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111000 con 0x7f13c8106a80 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35340/0 (socket says 172.21.15.38:35340) 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 -- 172.21.15.38:0/1553920653 learned_addr learned my addr 172.21.15.38:0/1553920653 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13cea5e640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8106a80 0x7f13c81a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13cda5c640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8101420 0x7f13c8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 -- 172.21.15.38:0/1553920653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8106a80 msgr2=0x7f13c81a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8106a80 0x7f13c81a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 -- 172.21.15.38:0/1553920653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8101420 msgr2=0x7f13c8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8101420 0x7f13c8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 -- 172.21.15.38:0/1553920653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c81a4890 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13ce25d640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009230 tx=0x7f13b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b803f030 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c81a4a60 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b8004440 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.206+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8035070 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.207+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a4ec0 con 0x7f13c8100a80 2024-09-19T05:39:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.208+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c8100ee0 con 0x7f13c8100a80 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.209+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f13b8042030 con 0x7f13c8100a80 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.209+0000 7f13c77fe640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 0x7f13b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.210+0000 7f13cda5c640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 0x7f13b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.210+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f13b80f01f0 con 0x7f13c8100a80 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.210+0000 7f13cda5c640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 0x7f13b00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13c806b2e0 tx=0x7f13bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.213+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13b80bc720 con 0x7f13c8100a80 2024-09-19T05:39:39.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:39 smithi038 ceph-mon[31227]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:39.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.395+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c8108ff0 con 0x7f13c8100a80 2024-09-19T05:39:39.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.396+0000 7f13c77fe640 1 -- 172.21.15.38:0/1553920653 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7f13b80c0a20 con 0x7f13c8100a80 2024-09-19T05:39:39.398 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:39.398 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 msgr2=0x7f13b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 0x7f13b00a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13c806b2e0 tx=0x7f13bc005f90 comp rx=0 tx=0).stop 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 msgr2=0x7f13c81992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009230 tx=0x7f13b8004040 comp rx=0 tx=0).stop 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 shutdown_connections 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13b00a3240 0x7f13b00a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c8106a80 0x7f13c81a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13c8101420 0x7f13c8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 --2- 172.21.15.38:0/1553920653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c8100a80 0x7f13c81992d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.398+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 >> 172.21.15.38:0/1553920653 conn(0x7f13c80fc7a0 msgr2=0x7f13c80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.399+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 shutdown_connections 2024-09-19T05:39:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:39.399+0000 7f13d04c1640 1 -- 172.21.15.38:0/1553920653 wait complete. 2024-09-19T05:39:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:39 smithi179 ceph-mon[35781]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:39.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:39 smithi047 ceph-mon[35327]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T05:39:40.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T05:39:40.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:40 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1553920653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:40.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:40 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1553920653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:40.462 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:40.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:40 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1553920653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:39:40.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 -- 172.21.15.38:0/140623270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38101de0 msgr2=0x7eff381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:40.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 --2- 172.21.15.38:0/140623270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38101de0 0x7eff381021c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff34009900 tx=0x7eff3402a410 comp rx=0 tx=0).stop 2024-09-19T05:39:40.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 -- 172.21.15.38:0/140623270 shutdown_connections 2024-09-19T05:39:40.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 --2- 172.21.15.38:0/140623270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38106f10 0x7eff3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:40.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 --2- 172.21.15.38:0/140623270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38102790 0x7eff381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 --2- 172.21.15.38:0/140623270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38101de0 0x7eff381021c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 -- 172.21.15.38:0/140623270 >> 172.21.15.38:0/140623270 conn(0x7eff380fd8c0 msgr2=0x7eff380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 -- 172.21.15.38:0/140623270 shutdown_connections 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.965+0000 7eff4060f640 1 -- 172.21.15.38:0/140623270 wait complete. 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 Processor -- start 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 -- start start 2024-09-19T05:39:40.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38101de0 0x7eff38198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38102790 0x7eff38198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff38111f70 con 0x7eff38106f10 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7eff38111df0 con 0x7eff38101de0 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff4060f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7eff381120f0 con 0x7eff38102790 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35370/0 (socket says 172.21.15.38:35370) 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 -- 172.21.15.38:0/3704964444 learned_addr learned my addr 172.21.15.38:0/3704964444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3dbaa640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38102790 0x7eff38198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3e3ab640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38101de0 0x7eff38198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 -- 172.21.15.38:0/3704964444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38102790 msgr2=0x7eff38198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:40.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38102790 0x7eff38198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 -- 172.21.15.38:0/3704964444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38101de0 msgr2=0x7eff38198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38101de0 0x7eff38198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.966+0000 7eff3ebac640 1 -- 172.21.15.38:0/3704964444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3819d3e0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff3ebac640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff3000eaa0 tx=0x7eff3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000cdb0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30005020 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3819d6d0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff300116b0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.967+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a4fb0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.968+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff381021c0 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.969+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7eff30005560 con 0x7eff38106f10 2024-09-19T05:39:40.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.970+0000 7eff2f7fe640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 0x7eff240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:40.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.970+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7eff300cfd40 con 0x7eff38106f10 2024-09-19T05:39:40.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.970+0000 7eff3e3ab640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 0x7eff240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:40.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.970+0000 7eff3e3ab640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 0x7eff240a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff34005ab0 tx=0x7eff34005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:40.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:40.974+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff30016030 con 0x7eff38106f10 2024-09-19T05:39:41.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:41.190+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7eff3802d7c0 con 0x7eff38106f10 2024-09-19T05:39:41.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:41 smithi179 ceph-mon[35781]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:41.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:41 smithi047 ceph-mon[35327]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:41.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:41 smithi038 ceph-mon[31227]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:42.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.153+0000 7eff2f7fe640 1 -- 172.21.15.38:0/3704964444 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7eff3009c820 con 0x7eff38106f10 2024-09-19T05:39:42.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 msgr2=0x7eff240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 0x7eff240a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff34005ab0 tx=0x7eff34005a20 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 msgr2=0x7eff3819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff3000eaa0 tx=0x7eff3000ef60 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 shutdown_connections 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7eff240a32c0 0x7eff240a5780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff38106f10 0x7eff3819cd00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff38102790 0x7eff38198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 --2- 172.21.15.38:0/3704964444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff38101de0 0x7eff38198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 >> 172.21.15.38:0/3704964444 conn(0x7eff380fd8c0 msgr2=0x7eff380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 shutdown_connections 2024-09-19T05:39:42.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:42.156+0000 7eff4060f640 1 -- 172.21.15.38:0/3704964444 wait complete. 2024-09-19T05:39:42.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:42 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:39:42.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:42 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:39:42.476 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:42 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:39:42.945 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T05:39:42.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T05:39:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:43.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:43 smithi038 ceph-mon[31227]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:43.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:43 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:39:43.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:43 smithi038 ceph-mon[31227]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:43 smithi179 ceph-mon[35781]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:43 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:39:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:43 smithi179 ceph-mon[35781]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:43.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:43 smithi047 ceph-mon[35327]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T05:39:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:43 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3704964444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:39:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:43 smithi047 ceph-mon[35327]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 -- 172.21.15.38:0/544890946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0106f10 msgr2=0x7f28a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 --2- 172.21.15.38:0/544890946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0106f10 0x7f28a010ea90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2898009ca0 tx=0x7f289802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 -- 172.21.15.38:0/544890946 shutdown_connections 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 --2- 172.21.15.38:0/544890946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0106f10 0x7f28a010ea90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 --2- 172.21.15.38:0/544890946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0102790 0x7f28a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.657+0000 7f28a73f5640 1 --2- 172.21.15.38:0/544890946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0101de0 0x7f28a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.658+0000 7f28a73f5640 1 -- 172.21.15.38:0/544890946 >> 172.21.15.38:0/544890946 conn(0x7f28a00fd8c0 msgr2=0x7f28a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.658+0000 7f28a73f5640 1 -- 172.21.15.38:0/544890946 shutdown_connections 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.658+0000 7f28a73f5640 1 -- 172.21.15.38:0/544890946 wait complete. 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.658+0000 7f28a73f5640 1 Processor -- start 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.658+0000 7f28a73f5640 1 -- start start 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0102790 0x7f28a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 0x7f28a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0111fc0 con 0x7f28a0101de0 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0111e40 con 0x7f28a0106f10 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a73f5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0112140 con 0x7f28a0102790 2024-09-19T05:39:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35380/0 (socket says 172.21.15.38:35380) 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 -- 172.21.15.38:0/649132363 learned_addr learned my addr 172.21.15.38:0/649132363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a4990640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0102790 0x7f28a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5992640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 0x7f28a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 -- 172.21.15.38:0/649132363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0102790 msgr2=0x7f28a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0102790 0x7f28a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 -- 172.21.15.38:0/649132363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 msgr2=0x7f28a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 0x7f28a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 -- 172.21.15.38:0/649132363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a01a6890 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5992640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 0x7f28a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28a5191640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c970 tx=0x7f289c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.659+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c007bb0 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.660+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c007d50 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.660+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a01a6b80 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.660+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00f450 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.660+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a01a7070 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.661+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a01021c0 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.662+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f289c005020 con 0x7f28a0101de0 2024-09-19T05:39:43.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.663+0000 7f28967fc640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 0x7f288c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.663+0000 7f28a4990640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 0x7f288c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.663+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f289c0cf940 con 0x7f28a0101de0 2024-09-19T05:39:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.663+0000 7f28a4990640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 0x7f288c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28a0075f80 tx=0x7f289000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:43.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.667+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f289c014030 con 0x7f28a0101de0 2024-09-19T05:39:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:43.882+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f28a0109bc0 con 0x7f28a0101de0 2024-09-19T05:39:44.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.071+0000 7f28967fc640 1 -- 172.21.15.38:0/649132363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f289c09c420 con 0x7f28a0101de0 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 msgr2=0x7f288c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 0x7f288c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28a0075f80 tx=0x7f289000b410 comp rx=0 tx=0).stop 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 msgr2=0x7f28a00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c970 tx=0x7f289c00ce30 comp rx=0 tx=0).stop 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 shutdown_connections 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f288c0a3240 0x7f288c0a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a0106f10 0x7f28a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28a0102790 0x7f28a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:44.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 --2- 172.21.15.38:0/649132363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28a0101de0 0x7f28a00744f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:44.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 >> 172.21.15.38:0/649132363 conn(0x7f28a00fd8c0 msgr2=0x7f28a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:44.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 shutdown_connections 2024-09-19T05:39:44.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:44.074+0000 7f28a73f5640 1 -- 172.21.15.38:0/649132363 wait complete. 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T05:39:44.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:44 smithi038 ceph-mon[31227]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T05:39:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:39:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:39:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:39:44.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:39:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T05:39:44.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:44 smithi179 ceph-mon[35781]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/649132363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-19T05:39:44.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:44 smithi047 ceph-mon[35327]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-19T05:39:44.862 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T05:39:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:active 2024-09-19T05:39:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:39:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T05:39:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:39:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:45 smithi179 ceph-mon[35781]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:active 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:39:45.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:45 smithi047 ceph-mon[35327]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:39:45.543 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:39:45.543 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] up:active 2024-09-19T05:39:45.543 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:39:45.544 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-19T05:39:45.544 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-19T05:39:45.544 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:45 smithi038 ceph-mon[31227]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 -- 172.21.15.38:0/1612146536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 msgr2=0x7fc3dc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/1612146536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc103180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009900 tx=0x7fc3cc02a480 comp rx=0 tx=0).stop 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 -- 172.21.15.38:0/1612146536 shutdown_connections 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/1612146536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 0x7fc3dc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/1612146536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 0x7fc3dc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/1612146536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc103180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 -- 172.21.15.38:0/1612146536 >> 172.21.15.38:0/1612146536 conn(0x7fc3dc0fc7a0 msgr2=0x7fc3dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 -- 172.21.15.38:0/1612146536 shutdown_connections 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.627+0000 7fc3e2f19640 1 -- 172.21.15.38:0/1612146536 wait complete. 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 Processor -- start 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 -- start start 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 0x7fc3dc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 0x7fc3dc101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc111f60 con 0x7fc3dc102da0 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc111de0 con 0x7fc3dc1036c0 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e2f19640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1120e0 con 0x7fc3dc10c180 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e0cb5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e0cb5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35400/0 (socket says 172.21.15.38:35400) 2024-09-19T05:39:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.628+0000 7fc3e0cb5640 1 -- 172.21.15.38:0/485878489 learned_addr learned my addr 172.21.15.38:0/485878489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e14b6640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 0x7fc3dc101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3dbfff640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 0x7fc3dc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 -- 172.21.15.38:0/485878489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 msgr2=0x7fc3dc101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 0x7fc3dc101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 -- 172.21.15.38:0/485878489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 msgr2=0x7fc3dc100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 0x7fc3dc100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 -- 172.21.15.38:0/485878489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc1a6900 con 0x7fc3dc102da0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e0cb5640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc006010 tx=0x7fc3cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc03f030 con 0x7fc3dc102da0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc1a6b90 con 0x7fc3dc102da0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc003ce0 con 0x7fc3dc102da0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.629+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc035070 con 0x7fc3dc102da0 2024-09-19T05:39:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.630+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a70c0 con 0x7fc3dc102da0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.631+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc103180 con 0x7fc3dc102da0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.633+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc3cc042030 con 0x7fc3dc102da0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.633+0000 7fc3d9ffb640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 0x7fc3c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.633+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc3cc0f03b0 con 0x7fc3dc102da0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.633+0000 7fc3dbfff640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 0x7fc3c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.634+0000 7fc3dbfff640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 0x7fc3c80a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc102490 tx=0x7fc3c400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.637+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3cc0bcf50 con 0x7fc3dc102da0 2024-09-19T05:39:45.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:45.849+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc3dc101f30 con 0x7fc3dc102da0 2024-09-19T05:39:46.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.083+0000 7fc3d9ffb640 1 -- 172.21.15.38:0/485878489 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fc3dc101f30 con 0x7fc3dc102da0 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 msgr2=0x7fc3c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 0x7fc3c80a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc102490 tx=0x7fc3c400a4c0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 msgr2=0x7fc3dc100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc006010 tx=0x7fc3cc0042f0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 shutdown_connections 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc3c80a3240 0x7fc3c80a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3dc10c180 0x7fc3dc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3dc1036c0 0x7fc3dc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 --2- 172.21.15.38:0/485878489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3dc102da0 0x7fc3dc100a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 >> 172.21.15.38:0/485878489 conn(0x7fc3dc0fc7a0 msgr2=0x7fc3dc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:46.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 shutdown_connections 2024-09-19T05:39:46.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:46.086+0000 7fc3e2f19640 1 -- 172.21.15.38:0/485878489 wait complete. 2024-09-19T05:39:46.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:active 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T05:39:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:46 smithi179 ceph-mon[35781]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:active 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T05:39:46.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:46 smithi047 ceph-mon[35327]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T05:39:46.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:39:46.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/485878489' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:39:46.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] up:active 2024-09-19T05:39:46.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:39:46.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-19T05:39:46.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-19T05:39:46.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:46 smithi038 ceph-mon[31227]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-19T05:39:46.856 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T05:39:47.122 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:active 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: Cluster is now healthy 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T05:39:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:39:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:47 smithi179 ceph-mon[35781]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T05:39:47.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T05:39:47.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:active 2024-09-19T05:39:47.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:39:47.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:39:47.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: Cluster is now healthy 2024-09-19T05:39:47.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T05:39:47.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:39:47.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:47 smithi047 ceph-mon[35327]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T05:39:47.538 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] up:active 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: Cluster is now healthy 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:39:47.539 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:47 smithi038 ceph-mon[31227]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-19T05:39:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 -- 172.21.15.38:0/2572290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370101420 msgr2=0x7f837010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 --2- 172.21.15.38:0/2572290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370101420 0x7f837010d950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8368009cd0 tx=0x7f836802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 -- 172.21.15.38:0/2572290440 shutdown_connections 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 --2- 172.21.15.38:0/2572290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370101420 0x7f837010d950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 --2- 172.21.15.38:0/2572290440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370100a80 0x7f8370100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 --2- 172.21.15.38:0/2572290440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370106a80 0x7f8370106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 -- 172.21.15.38:0/2572290440 >> 172.21.15.38:0/2572290440 conn(0x7f83700fc7a0 msgr2=0x7f83700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 -- 172.21.15.38:0/2572290440 shutdown_connections 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.622+0000 7f837502d640 1 -- 172.21.15.38:0/2572290440 wait complete. 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 Processor -- start 2024-09-19T05:39:47.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 -- start start 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370100a80 0x7f83700fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 0x7f83700fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8370110ea0 con 0x7f8370106a80 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8370110d20 con 0x7f8370101420 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f837502d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8370111020 con 0x7f8370100a80 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f836f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f836f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46170/0 (socket says 172.21.15.38:46170) 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.623+0000 7f836f59e640 1 -- 172.21.15.38:0/3067868996 learned_addr learned my addr 172.21.15.38:0/3067868996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836659c640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 0x7f83700fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836ed9d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370100a80 0x7f83700fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 -- 172.21.15.38:0/3067868996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370100a80 msgr2=0x7f83700fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370100a80 0x7f83700fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 -- 172.21.15.38:0/3067868996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 msgr2=0x7f83700fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 0x7f83700fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 -- 172.21.15.38:0/3067868996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83701a6930 con 0x7f8370106a80 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836659c640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 0x7f83700fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836f59e640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8368006010 tx=0x7f8368003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368004450 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8368003e10 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83701a6bc0 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680466e0 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.624+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a7100 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.625+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8370100ee0 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.626+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f83680305a0 con 0x7f8370106a80 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.627+0000 7f836cd99640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 0x7f835c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.627+0000 7f836ed9d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 0x7f835c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:47.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.627+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f836802e030 con 0x7f8370106a80 2024-09-19T05:39:47.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.628+0000 7f836ed9d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 0x7f835c0a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8360009890 tx=0x7f8360008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:47.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.630+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f836804b060 con 0x7f8370106a80 2024-09-19T05:39:47.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:47.841+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f837010a020 con 0x7f8370106a80 2024-09-19T05:39:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.098+0000 7f836cd99640 1 -- 172.21.15.38:0/3067868996 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f8368044380 con 0x7f8370106a80 2024-09-19T05:39:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 msgr2=0x7f835c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 0x7f835c0a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8360009890 tx=0x7f8360008040 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 msgr2=0x7f83700fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8368006010 tx=0x7f8368003c20 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 shutdown_connections 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f835c0a3240 0x7f835c0a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8370106a80 0x7f83700fd7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8370101420 0x7f83700fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 --2- 172.21.15.38:0/3067868996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8370100a80 0x7f83700fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 >> 172.21.15.38:0/3067868996 conn(0x7f83700fc7a0 msgr2=0x7f837006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 shutdown_connections 2024-09-19T05:39:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:48.101+0000 7f837502d640 1 -- 172.21.15.38:0/3067868996 wait complete. 2024-09-19T05:39:48.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:48 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:39:48.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:48 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:39:48.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:48 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:active 2024-09-19T05:39:48.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:48 smithi038 ceph-mon[31227]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:39:48.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:48 smithi038 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:39:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:48 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:39:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:48 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:39:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:48 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:active 2024-09-19T05:39:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:48 smithi179 ceph-mon[35781]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:39:48.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:48 smithi179 ceph-mon[35781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:39:48.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:48 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:39:48.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:48 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3067868996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:39:48.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:48 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] up:active 2024-09-19T05:39:48.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:48 smithi047 ceph-mon[35327]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:39:48.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:48 smithi047 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:39:48.836 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-19T05:39:48.836 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T05:39:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:49 smithi179 ceph-mon[35781]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-09-19T05:39:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:49 smithi047 ceph-mon[35327]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-09-19T05:39:49.481 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-mon[31227]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-09-19T05:39:49.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- 172.21.15.38:0/3676618157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d41027a0 msgr2=0x7f23d41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 --2- 172.21.15.38:0/3676618157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d41027a0 0x7f23d41067f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009900 tx=0x7f23c802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- 172.21.15.38:0/3676618157 shutdown_connections 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 --2- 172.21.15.38:0/3676618157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 0x7f23d410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 --2- 172.21.15.38:0/3676618157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d41027a0 0x7f23d41067f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 --2- 172.21.15.38:0/3676618157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d4101df0 0x7f23d41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- 172.21.15.38:0/3676618157 >> 172.21.15.38:0/3676618157 conn(0x7f23d40fd8b0 msgr2=0x7f23d40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- 172.21.15.38:0/3676618157 shutdown_connections 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- 172.21.15.38:0/3676618157 wait complete. 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 Processor -- start 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.565+0000 7f23d935b640 1 -- start start 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d41027a0 0x7f23d4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 0x7f23d419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4111f80 con 0x7f23d4101df0 2024-09-19T05:39:49.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4111e00 con 0x7f23d41027a0 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d935b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4112100 con 0x7f23d4106f20 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46192/0 (socket says 172.21.15.38:46192) 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 -- 172.21.15.38:0/3857816323 learned_addr learned my addr 172.21.15.38:0/3857816323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d27fc640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d41027a0 0x7f23d4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d37fe640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 0x7f23d419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 -- 172.21.15.38:0/3857816323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 msgr2=0x7f23d419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:49.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 0x7f23d419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 -- 172.21.15.38:0/3857816323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d41027a0 msgr2=0x7f23d4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d41027a0 0x7f23d4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 -- 172.21.15.38:0/3857816323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d419d3d0 con 0x7f23d4101df0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.566+0000 7f23d2ffd640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23c400d950 tx=0x7f23c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.567+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4017070 con 0x7f23d4101df0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.567+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d419d6c0 con 0x7f23d4101df0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.567+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c4009c30 con 0x7f23d4101df0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.567+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c400f6b0 con 0x7f23d4101df0 2024-09-19T05:39:49.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.567+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41a4fa0 con 0x7f23d4101df0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.568+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d41021d0 con 0x7f23d4101df0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.569+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f23c40109d0 con 0x7f23d4101df0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.570+0000 7f23bbfff640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 0x7f23bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.570+0000 7f23d27fc640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 0x7f23bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.570+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f23c40cf6d0 con 0x7f23d4101df0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.570+0000 7f23d27fc640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 0x7f23bc0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009230 tx=0x7f23c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:49.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.573+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23c4009de0 con 0x7f23d4101df0 2024-09-19T05:39:49.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.773+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f23d4062630 con 0x7f23bc0a3240 2024-09-19T05:39:49.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.808+0000 7f23bbfff640 1 -- 172.21.15.38:0/3857816323 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f23d4062630 con 0x7f23bc0a3240 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 msgr2=0x7f23bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 0x7f23bc0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009230 tx=0x7f23c80058d0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 msgr2=0x7f23d4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23c400d950 tx=0x7f23c400de10 comp rx=0 tx=0).stop 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 shutdown_connections 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f23bc0a3240 0x7f23bc0a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23d4106f20 0x7f23d419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23d41027a0 0x7f23d4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 --2- 172.21.15.38:0/3857816323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d4101df0 0x7f23d4198420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:49.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 >> 172.21.15.38:0/3857816323 conn(0x7f23d40fd8b0 msgr2=0x7f23d40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:49.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 shutdown_connections 2024-09-19T05:39:49.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:49.810+0000 7f23d935b640 1 -- 172.21.15.38:0/3857816323 wait complete. 2024-09-19T05:39:50.128 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:39:49.778+0000 7f38d30c9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:39:50.128 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:39:49.778+0000 7f38d30c9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:39:50.128 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:39:49.778+0000 7f38d30c9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:39:50.128 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:39:49.778+0000 7f38d30c9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:39:50.128 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:39:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:39:49.778+0000 7f38d30c9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:39:50.453 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T05:39:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 -- 172.21.15.38:0/3081796602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 msgr2=0x7f3328100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 --2- 172.21.15.38:0/3081796602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f3328100cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3318009900 tx=0x7f331802a410 comp rx=0 tx=0).stop 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 -- 172.21.15.38:0/3081796602 shutdown_connections 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 --2- 172.21.15.38:0/3081796602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 0x7f3328110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 --2- 172.21.15.38:0/3081796602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 0x7f332810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 --2- 172.21.15.38:0/3081796602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f3328100cb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 -- 172.21.15.38:0/3081796602 >> 172.21.15.38:0/3081796602 conn(0x7f33280fc7a0 msgr2=0x7f33280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 -- 172.21.15.38:0/3081796602 shutdown_connections 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.163+0000 7f332e9a4640 1 -- 172.21.15.38:0/3081796602 wait complete. 2024-09-19T05:39:51.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 Processor -- start 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 -- start start 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 0x7f332819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 0x7f332819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f33281141b0 con 0x7f33281008d0 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3328114030 con 0x7f332810e480 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332e9a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3328114330 con 0x7f3328101280 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46218/0 (socket says 172.21.15.38:46218) 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 -- 172.21.15.38:0/2824904501 learned_addr learned my addr 172.21.15.38:0/2824904501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f331ffff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 0x7f332819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f332cf41640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 0x7f332819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:51.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 -- 172.21.15.38:0/2824904501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 msgr2=0x7f332819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 0x7f332819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 -- 172.21.15.38:0/2824904501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 msgr2=0x7f332819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 0x7f332819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.164+0000 7f3327fff640 1 -- 172.21.15.38:0/2824904501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332819f640 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f3327fff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3318006010 tx=0x7f3318004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331803f030 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3318004440 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332819f8d0 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318035070 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.165+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281a7210 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.166+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328100cb0 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.167+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3318042030 con 0x7f33281008d0 2024-09-19T05:39:51.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.168+0000 7f3325ffb640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 0x7f33140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.168+0000 7f331ffff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 0x7f33140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.168+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f33180f0210 con 0x7f33281008d0 2024-09-19T05:39:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.168+0000 7f331ffff640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 0x7f33140a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f332810ed80 tx=0x7f3310005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:51.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.171+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33180bcd90 con 0x7f33281008d0 2024-09-19T05:39:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.375+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3328062630 con 0x7f33140a32c0 2024-09-19T05:39:51.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.401+0000 7f3325ffb640 1 -- 172.21.15.38:0/2824904501 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3328062630 con 0x7f33140a32c0 2024-09-19T05:39:51.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 msgr2=0x7f33140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 0x7f33140a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f332810ed80 tx=0x7f3310005f90 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 msgr2=0x7f332819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3318006010 tx=0x7f3318004040 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 shutdown_connections 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f33140a32c0 0x7f33140a5780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f332810e480 0x7f332819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3328101280 0x7f332819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 --2- 172.21.15.38:0/2824904501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33281008d0 0x7f332819a690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 >> 172.21.15.38:0/2824904501 conn(0x7f33280fc7a0 msgr2=0x7f33280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 shutdown_connections 2024-09-19T05:39:51.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:51.403+0000 7f332e9a4640 1 -- 172.21.15.38:0/2824904501 wait complete. 2024-09-19T05:39:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:51 smithi179 ceph-mon[35781]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T05:39:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:51 smithi179 ceph-mon[35781]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:51 smithi047 ceph-mon[35327]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T05:39:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:51 smithi047 ceph-mon[35327]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:51 smithi038 ceph-mon[31227]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T05:39:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:51 smithi038 ceph-mon[31227]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:52.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T05:39:52.469 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:52.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:52 smithi047 ceph-mon[35327]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:52.590 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:52 smithi038 ceph-mon[31227]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:52.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:52 smithi179 ceph-mon[35781]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 -- 172.21.15.38:0/1048528279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c100e20 msgr2=0x7fee3c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 --2- 172.21.15.38:0/1048528279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c100e20 0x7fee3c101200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a480 comp rx=0 tx=0).stop 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 -- 172.21.15.38:0/1048528279 shutdown_connections 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 --2- 172.21.15.38:0/1048528279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 --2- 172.21.15.38:0/1048528279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c102570 0x7fee3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 --2- 172.21.15.38:0/1048528279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c100e20 0x7fee3c101200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 -- 172.21.15.38:0/1048528279 >> 172.21.15.38:0/1048528279 conn(0x7fee3c0fc7d0 msgr2=0x7fee3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 -- 172.21.15.38:0/1048528279 shutdown_connections 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.933+0000 7fee44070640 1 -- 172.21.15.38:0/1048528279 wait complete. 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 Processor -- start 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 -- start start 2024-09-19T05:39:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c100e20 0x7fee3c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c110f80 con 0x7fee3c102570 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c110e00 con 0x7fee3c102ef0 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee44070640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111100 con 0x7fee3c100e20 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46246/0 (socket says 172.21.15.38:46246) 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 -- 172.21.15.38:0/1134613272 learned_addr learned my addr 172.21.15.38:0/1134613272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee41e0c640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c100e20 0x7fee3c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4260d640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 -- 172.21.15.38:0/1134613272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c100e20 msgr2=0x7fee3c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c100e20 0x7fee3c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 -- 172.21.15.38:0/1134613272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 msgr2=0x7fee3c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.934+0000 7fee4160b640 1 -- 172.21.15.38:0/1134613272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c19d480 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee4260d640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee4160b640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d770 tx=0x7fee2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c017070 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c005500 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c19d770 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c004e00 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.935+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c1a5050 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.936+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c101200 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.937+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fee2c004fa0 con 0x7fee3c102570 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.938+0000 7fee32ffd640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.938+0000 7fee41e0c640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.938+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fee2c0cf810 con 0x7fee3c102570 2024-09-19T05:39:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.938+0000 7fee41e0c640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 0x7fee280a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee38005ab0 tx=0x7fee38005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:52.942+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee2c09c190 con 0x7fee3c102570 2024-09-19T05:39:53.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.145+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fee3c062630 con 0x7fee280a32d0 2024-09-19T05:39:53.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.150+0000 7fee32ffd640 1 -- 172.21.15.38:0/1134613272 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fee3c062630 con 0x7fee280a32d0 2024-09-19T05:39:53.152 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_0/3c228e6f-5971-4c80-8910-48b749696d90 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 msgr2=0x7fee280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 0x7fee280a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee38005ab0 tx=0x7fee38005a20 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 msgr2=0x7fee3c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d770 tx=0x7fee2c00dc30 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 shutdown_connections 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee280a32d0 0x7fee280a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c102ef0 0x7fee3c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee3c102570 0x7fee3c198a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 --2- 172.21.15.38:0/1134613272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee3c100e20 0x7fee3c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 >> 172.21.15.38:0/1134613272 conn(0x7fee3c0fc7d0 msgr2=0x7fee3c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 shutdown_connections 2024-09-19T05:39:53.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:53.152+0000 7fee44070640 1 -- 172.21.15.38:0/1134613272 wait complete. 2024-09-19T05:39:53.469 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:53 smithi038 ceph-mon[31227]: pgmap v308: 129 pgs: 129 active+clean; 607 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-09-19T05:39:53.469 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:53 smithi038 ceph-mon[31227]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:39:53.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:53 smithi047 ceph-mon[35327]: pgmap v308: 129 pgs: 129 active+clean; 607 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-09-19T05:39:53.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:53 smithi047 ceph-mon[35327]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:39:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:53 smithi179 ceph-mon[35781]: pgmap v308: 129 pgs: 129 active+clean; 607 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-09-19T05:39:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:53 smithi179 ceph-mon[35781]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:39:53.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T05:39:54.119 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:54.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:54 smithi047 ceph-mon[35327]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:54.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:54 smithi038 ceph-mon[31227]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 -- 172.21.15.38:0/1017041794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94108300 msgr2=0x7fdb9410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 --2- 172.21.15.38:0/1017041794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94108300 0x7fdb9410ec70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009cd0 tx=0x7fdb8c02a8f0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 -- 172.21.15.38:0/1017041794 shutdown_connections 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 --2- 172.21.15.38:0/1017041794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94108300 0x7fdb9410ec70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 --2- 172.21.15.38:0/1017041794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94103780 0x7fdb94107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 --2- 172.21.15.38:0/1017041794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94102dd0 0x7fdb941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 -- 172.21.15.38:0/1017041794 >> 172.21.15.38:0/1017041794 conn(0x7fdb940fc7d0 msgr2=0x7fdb940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 -- 172.21.15.38:0/1017041794 shutdown_connections 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.584+0000 7fdb9b124640 1 -- 172.21.15.38:0/1017041794 wait complete. 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 Processor -- start 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 -- start start 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94102dd0 0x7fdb941986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 0x7fdb9419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94112170 con 0x7fdb94103780 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94111ff0 con 0x7fdb94102dd0 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb9b124640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941122f0 con 0x7fdb94108300 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46260/0 (socket says 172.21.15.38:46260) 2024-09-19T05:39:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb93fff640 1 -- 172.21.15.38:0/3314662797 learned_addr learned my addr 172.21.15.38:0/3314662797 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb98ec0640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94102dd0 0x7fdb941986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.585+0000 7fdb996c1640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 0x7fdb9419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 -- 172.21.15.38:0/3314662797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 msgr2=0x7fdb9419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 0x7fdb9419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 -- 172.21.15.38:0/3314662797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94102dd0 msgr2=0x7fdb941986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94102dd0 0x7fdb941986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 -- 172.21.15.38:0/3314662797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9419d660 con 0x7fdb94103780 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb996c1640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 0x7fdb9419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb93fff640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006fd0 tx=0x7fdb8000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80011830 con 0x7fdb94103780 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb80016070 con 0x7fdb94103780 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9419d950 con 0x7fdb94103780 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000f2e0 con 0x7fdb94103780 2024-09-19T05:39:54.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.586+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941a5230 con 0x7fdb94103780 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.587+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94103230 con 0x7fdb94103780 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.588+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdb8001b460 con 0x7fdb94103780 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.589+0000 7fdb91ffb640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 0x7fdb840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.589+0000 7fdb98ec0640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 0x7fdb840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.589+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fdb800cf270 con 0x7fdb94103780 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.589+0000 7fdb98ec0640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 0x7fdb840a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009890 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.593+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb8009bd00 con 0x7fdb94103780 2024-09-19T05:39:54.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:54 smithi179 ceph-mon[35781]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:54.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.795+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdb94062630 con 0x7fdb840a3350 2024-09-19T05:39:54.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.822+0000 7fdb91ffb640 1 -- 172.21.15.38:0/3314662797 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb94062630 con 0x7fdb840a3350 2024-09-19T05:39:54.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 msgr2=0x7fdb840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:54.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 0x7fdb840a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009890 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2024-09-19T05:39:54.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 msgr2=0x7fdb94198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:54.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006fd0 tx=0x7fdb8000b8e0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 shutdown_connections 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdb840a3350 0x7fdb840a5810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb94108300 0x7fdb9419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb94103780 0x7fdb94198bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 --2- 172.21.15.38:0/3314662797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb94102dd0 0x7fdb941986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.824+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 >> 172.21.15.38:0/3314662797 conn(0x7fdb940fc7d0 msgr2=0x7fdb94105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.825+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 shutdown_connections 2024-09-19T05:39:54.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:54.825+0000 7fdb9b124640 1 -- 172.21.15.38:0/3314662797 wait complete. 2024-09-19T05:39:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:55 smithi038 ceph-mon[31227]: pgmap v309: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-19T05:39:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:55 smithi179 ceph-mon[35781]: pgmap v309: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-19T05:39:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:55 smithi047 ceph-mon[35327]: pgmap v309: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-19T05:39:55.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T05:39:56.024 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:56.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:56 smithi038 ceph-mon[31227]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 -- 172.21.15.38:0/1215812757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4101de0 msgr2=0x7f68b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/1215812757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4101de0 0x7f68b41021c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009900 tx=0x7f68b002a410 comp rx=0 tx=0).stop 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 -- 172.21.15.38:0/1215812757 shutdown_connections 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/1215812757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4106f10 0x7f68b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/1215812757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 0x7f68b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/1215812757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4101de0 0x7f68b41021c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 -- 172.21.15.38:0/1215812757 >> 172.21.15.38:0/1215812757 conn(0x7f68b40fd8c0 msgr2=0x7f68b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 -- 172.21.15.38:0/1215812757 shutdown_connections 2024-09-19T05:39:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.597+0000 7f68bc2ff640 1 -- 172.21.15.38:0/1215812757 wait complete. 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 Processor -- start 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 -- start start 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4101de0 0x7f68b4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 0x7f68b4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111f30 con 0x7f68b4106f10 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111db0 con 0x7f68b4102790 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68bc2ff640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41120b0 con 0x7f68b4101de0 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68ba89c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68ba89c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46284/0 (socket says 172.21.15.38:46284) 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68ba89c640 1 -- 172.21.15.38:0/3586637214 learned_addr learned my addr 172.21.15.38:0/3586637214 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.598+0000 7f68b989a640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 0x7f68b4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba09b640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4101de0 0x7f68b4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:56.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 -- 172.21.15.38:0/3586637214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4101de0 msgr2=0x7f68b4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4101de0 0x7f68b4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 -- 172.21.15.38:0/3586637214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 msgr2=0x7f68b4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 0x7f68b4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 -- 172.21.15.38:0/3586637214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b419d4c0 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ba89c640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eaa0 tx=0x7f68ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00cdb0 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b419d7b0 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac005020 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0116b0 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.599+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a5090 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.600+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b41021c0 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.601+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f68ac005560 con 0x7f68b4106f10 2024-09-19T05:39:56.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.602+0000 7f68ab7fe640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 0x7f68a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.602+0000 7f68ba09b640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 0x7f68a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.602+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f68ac0d0200 con 0x7f68b4106f10 2024-09-19T05:39:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.603+0000 7f68ba09b640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 0x7f68a00a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005ab0 tx=0x7f68b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.605+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68ac016030 con 0x7f68b4106f10 2024-09-19T05:39:56.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:56 smithi179 ceph-mon[35781]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:56.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:56 smithi047 ceph-mon[35327]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:56.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.809+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f68b410a160 con 0x7f68a00a32d0 2024-09-19T05:39:56.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.813+0000 7f68ab7fe640 1 -- 172.21.15.38:0/3586637214 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f68b00315b0 con 0x7f68a00a32d0 2024-09-19T05:39:56.819 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_1/216dcc11-78f6-472d-8036-e412a733f802 2024-09-19T05:39:56.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.815+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 msgr2=0x7f68a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.815+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 0x7f68a00a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005ab0 tx=0x7f68b0005a20 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.815+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 msgr2=0x7f68b419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.815+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eaa0 tx=0x7f68ac00ef60 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 shutdown_connections 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f68a00a32d0 0x7f68a00a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68b4106f10 0x7f68b419cde0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68b4102790 0x7f68b4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 --2- 172.21.15.38:0/3586637214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68b4101de0 0x7f68b4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 >> 172.21.15.38:0/3586637214 conn(0x7f68b40fd8c0 msgr2=0x7f68b40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 shutdown_connections 2024-09-19T05:39:56.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:56.816+0000 7f68bc2ff640 1 -- 172.21.15.38:0/3586637214 wait complete. 2024-09-19T05:39:57.503 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:57 smithi038 ceph-mon[31227]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-19T05:39:57.503 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:57 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:39:57.504 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:39:57.688 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:57 smithi047 ceph-mon[35327]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-19T05:39:57.688 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:57 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:39:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:57 smithi179 ceph-mon[35781]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-19T05:39:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:57 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:39:58.402 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:39:58.527 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:58 smithi038 ceph-mon[31227]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:58.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:58 smithi179 ceph-mon[35781]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:58.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:58 smithi047 ceph-mon[35327]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:39:58.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.909+0000 7fb375975640 1 -- 172.21.15.38:0/4174898351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb370101e10 msgr2=0x7fb3701021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:58.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.909+0000 7fb375975640 1 --2- 172.21.15.38:0/4174898351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb370101e10 0x7fb3701021f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb36002a410 comp rx=0 tx=0).stop 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 -- 172.21.15.38:0/4174898351 shutdown_connections 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 --2- 172.21.15.38:0/4174898351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370106f40 0x7fb37010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 --2- 172.21.15.38:0/4174898351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3701027c0 0x7fb370106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 --2- 172.21.15.38:0/4174898351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb370101e10 0x7fb3701021f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 -- 172.21.15.38:0/4174898351 >> 172.21.15.38:0/4174898351 conn(0x7fb3700fd8d0 msgr2=0x7fb3700ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 -- 172.21.15.38:0/4174898351 shutdown_connections 2024-09-19T05:39:58.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 -- 172.21.15.38:0/4174898351 wait complete. 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 Processor -- start 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.910+0000 7fb375975640 1 -- start start 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370101e10 0x7fb370198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 0x7fb37019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb370111fa0 con 0x7fb3701027c0 2024-09-19T05:39:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb370111e20 con 0x7fb370106f40 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb375975640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb370112120 con 0x7fb370101e10 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47924/0 (socket says 172.21.15.38:47924) 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 -- 172.21.15.38:0/1179361021 learned_addr learned my addr 172.21.15.38:0/1179361021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36effd640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370101e10 0x7fb370198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36f7fe640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 0x7fb37019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 -- 172.21.15.38:0/1179361021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370101e10 msgr2=0x7fb370198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370101e10 0x7fb370198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 -- 172.21.15.38:0/1179361021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 msgr2=0x7fb37019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 0x7fb37019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 -- 172.21.15.38:0/1179361021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37019d410 con 0x7fb3701027c0 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36f7fe640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 0x7fb37019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb36e7fc640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:58.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.911+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364017070 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.912+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb364009c30 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.912+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37019d700 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.912+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400f6b0 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.912+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3701a4fe0 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.913+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3701021f0 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.914+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb3640109d0 con 0x7fb3701027c0 2024-09-19T05:39:58.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.915+0000 7fb374973640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 0x7fb3580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:39:58.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.915+0000 7fb36effd640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 0x7fb3580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:39:58.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.915+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb3640cf8d0 con 0x7fb3701027c0 2024-09-19T05:39:58.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.916+0000 7fb36effd640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 0x7fb3580a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb360009230 tx=0x7fb3600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:39:58.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:58.919+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb364009de0 con 0x7fb3701027c0 2024-09-19T05:39:59.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.159+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb37010a180 con 0x7fb3701027c0 2024-09-19T05:39:59.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.160+0000 7fb374973640 1 -- 172.21.15.38:0/1179361021 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7fb36409c250 con 0x7fb3701027c0 2024-09-19T05:39:59.163 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:39:59.163 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:39:59.163 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:39:59.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.162+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 msgr2=0x7fb3580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.162+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 0x7fb3580a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb360009230 tx=0x7fb3600058d0 comp rx=0 tx=0).stop 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 msgr2=0x7fb3701989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d950 tx=0x7fb36400de10 comp rx=0 tx=0).stop 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 shutdown_connections 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb3580a3350 0x7fb3580a5810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb370106f40 0x7fb37019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3701027c0 0x7fb3701989a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:59.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 --2- 172.21.15.38:0/1179361021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb370101e10 0x7fb370198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:39:59.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 >> 172.21.15.38:0/1179361021 conn(0x7fb3700fd8d0 msgr2=0x7fb3700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:39:59.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 shutdown_connections 2024-09-19T05:39:59.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:39:59.163+0000 7fb375975640 1 -- 172.21.15.38:0/1179361021 wait complete. 2024-09-19T05:39:59.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: pgmap v311: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-09-19T05:39:59.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1179361021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:59.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:39:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:39:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: pgmap v311: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1179361021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:59.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:39:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: pgmap v311: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1179361021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:39:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:39:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:40:00.125 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:00.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 -- 172.21.15.38:0/2111147172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800106f10 msgr2=0x7ff80010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 --2- 172.21.15.38:0/2111147172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800106f10 0x7ff80010ea90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009ca0 tx=0x7ff7f802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 -- 172.21.15.38:0/2111147172 shutdown_connections 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 --2- 172.21.15.38:0/2111147172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800106f10 0x7ff80010ea90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 --2- 172.21.15.38:0/2111147172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800102790 0x7ff8001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 --2- 172.21.15.38:0/2111147172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800101de0 0x7ff8001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 -- 172.21.15.38:0/2111147172 >> 172.21.15.38:0/2111147172 conn(0x7ff8000fd8c0 msgr2=0x7ff8000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 -- 172.21.15.38:0/2111147172 shutdown_connections 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.645+0000 7ff8065a2640 1 -- 172.21.15.38:0/2111147172 wait complete. 2024-09-19T05:40:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 Processor -- start 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 -- start start 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800101de0 0x7ff800198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800106f10 0x7ff80019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff800111f70 con 0x7ff800102790 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff800111df0 con 0x7ff800106f10 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.646+0000 7ff8065a2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff8001120f0 con 0x7ff800101de0 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47936/0 (socket says 172.21.15.38:47936) 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 -- 172.21.15.38:0/1693365357 learned_addr learned my addr 172.21.15.38:0/1693365357 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7fffff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800101de0 0x7ff800198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff804b3f640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800106f10 0x7ff80019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:00.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 -- 172.21.15.38:0/1693365357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800101de0 msgr2=0x7ff800198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800101de0 0x7ff800198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 -- 172.21.15.38:0/1693365357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800106f10 msgr2=0x7ff80019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800106f10 0x7ff80019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 -- 172.21.15.38:0/1693365357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80019d3c0 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7f7fff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d770 tx=0x7ff7e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8017070 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e8005500 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80019d6b0 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8004e00 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.647+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8001a4f90 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.648+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8001021c0 con 0x7ff800102790 2024-09-19T05:40:00.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.650+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff7e8004fa0 con 0x7ff800102790 2024-09-19T05:40:00.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.651+0000 7ff7fdffb640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 0x7ff7ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:00.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.651+0000 7ff7fffff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 0x7ff7ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.651+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7ff7e80cfa30 con 0x7ff800102790 2024-09-19T05:40:00.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.651+0000 7ff7fffff640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 0x7ff7ec0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009bd0 tx=0x7ff7f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:00.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.654+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7e809bd60 con 0x7ff800102790 2024-09-19T05:40:00.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.831+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff80010a150 con 0x7ff800102790 2024-09-19T05:40:00.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:40:00.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:40:00.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:00 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:40:00.838 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:00 smithi038 ceph-mon[31227]: overall HEALTH_OK 2024-09-19T05:40:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.833+0000 7ff7fdffb640 1 -- 172.21.15.38:0/1693365357 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7ff7e80a0060 con 0x7ff800102790 2024-09-19T05:40:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:00.839 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 msgr2=0x7ff7ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 0x7ff7ec0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009bd0 tx=0x7ff7f0005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 msgr2=0x7ff800198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d770 tx=0x7ff7e800dc30 comp rx=0 tx=0).stop 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 shutdown_connections 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7ec0a32d0 0x7ff7ec0a5790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff800106f10 0x7ff80019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff800102790 0x7ff800198950 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 --2- 172.21.15.38:0/1693365357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff800101de0 0x7ff800198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 >> 172.21.15.38:0/1693365357 conn(0x7ff8000fd8c0 msgr2=0x7ff8000feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 shutdown_connections 2024-09-19T05:40:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:00.835+0000 7ff8065a2640 1 -- 172.21.15.38:0/1693365357 wait complete. 2024-09-19T05:40:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:40:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:40:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:00 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:40:00.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:00 smithi179 ceph-mon[35781]: overall HEALTH_OK 2024-09-19T05:40:00.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:40:00.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:40:00.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:00 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:40:00.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:00 smithi047 ceph-mon[35327]: overall HEALTH_OK 2024-09-19T05:40:01.503 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:40:01.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:40:01.767 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:01.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:01 smithi038 ceph-mon[31227]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-09-19T05:40:01.889 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:01 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1693365357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:01 smithi179 ceph-mon[35781]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-09-19T05:40:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:01 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1693365357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:01.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:01 smithi047 ceph-mon[35327]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-09-19T05:40:01.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:01 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1693365357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:02.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 -- 172.21.15.38:0/3833752140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4101de0 msgr2=0x7fb4d41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/3833752140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4101de0 0x7fb4d41021c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009ca0 tx=0x7fb4c402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 -- 172.21.15.38:0/3833752140 shutdown_connections 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/3833752140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4106f10 0x7fb4d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/3833752140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 0x7fb4d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/3833752140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4101de0 0x7fb4d41021c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 -- 172.21.15.38:0/3833752140 >> 172.21.15.38:0/3833752140 conn(0x7fb4d40fd880 msgr2=0x7fb4d40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 -- 172.21.15.38:0/3833752140 shutdown_connections 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.246+0000 7fb4dc45e640 1 -- 172.21.15.38:0/3833752140 wait complete. 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 Processor -- start 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 -- start start 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4101de0 0x7fb4d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 0x7fb4d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4111f80 con 0x7fb4d4106f10 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4111e00 con 0x7fb4d4102790 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4dc45e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4112100 con 0x7fb4d4101de0 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47966/0 (socket says 172.21.15.38:47966) 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 -- 172.21.15.38:0/4192940575 learned_addr learned my addr 172.21.15.38:0/4192940575 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4d99f9640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 0x7fb4d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da1fa640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4101de0 0x7fb4d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 -- 172.21.15.38:0/4192940575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4101de0 msgr2=0x7fb4d40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4101de0 0x7fb4d40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 -- 172.21.15.38:0/4192940575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 msgr2=0x7fb4d4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 0x7fb4d4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.247+0000 7fb4da9fb640 1 -- 172.21.15.38:0/4192940575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d41a68f0 con 0x7fb4d4106f10 2024-09-19T05:40:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4da9fb640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00d750 tx=0x7fb4cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc005500 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41a6be0 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004d90 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.248+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a70d0 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.249+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d41021c0 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.250+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb4cc004f30 con 0x7fb4d4106f10 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.251+0000 7fb4d37fe640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 0x7fb4bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.251+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb4cc0cf760 con 0x7fb4d4106f10 2024-09-19T05:40:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.251+0000 7fb4da1fa640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 0x7fb4bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.252+0000 7fb4da1fa640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 0x7fb4bc0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009430 tx=0x7fb4c4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.255+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4cc09c080 con 0x7fb4d4106f10 2024-09-19T05:40:02.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.430+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4d4062630 con 0x7fb4bc0a32d0 2024-09-19T05:40:02.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.440+0000 7fb4d37fe640 1 -- 172.21.15.38:0/4192940575 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340381 (secure 0 0 0) 0x7fb4d4062630 con 0x7fb4bc0a32d0 2024-09-19T05:40:02.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:02.447 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 msgr2=0x7fb4bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 0x7fb4bc0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009430 tx=0x7fb4c4000f40 comp rx=0 tx=0).stop 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 msgr2=0x7fb4d4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00d750 tx=0x7fb4cc00dc10 comp rx=0 tx=0).stop 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 shutdown_connections 2024-09-19T05:40:02.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb4bc0a32d0 0x7fb4bc0a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d4106f10 0x7fb4d4074f70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d4102790 0x7fb4d4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 --2- 172.21.15.38:0/4192940575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4d4101de0 0x7fb4d40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 >> 172.21.15.38:0/4192940575 conn(0x7fb4d40fd880 msgr2=0x7fb4d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 shutdown_connections 2024-09-19T05:40:02.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:02.443+0000 7fb4dc45e640 1 -- 172.21.15.38:0/4192940575 wait complete. 2024-09-19T05:40:03.212 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-19T05:40:01.597163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650495,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":293,"num_read_kb":265,"num_write":428,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354540,"kb_used_data":13612,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587284,"statfs":{"total":1151940427776,"available":1151577378816,"internally_reserved":0,"allocated":13938688,"data_stored":7668963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50651,"num_objects":14,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":145,"num_read_kb":136,"num_write":65,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003452"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:00.129056+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:40:00.129056+0000","last_peered":"2024-09-19T05:40:00.129056+0000","last_clean":"2024-09-19T05:40:00.129056+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:40:00.129056+0000","last_undegraded":"2024-09-19T05:40:00.129056+0000","last_fullsized":"2024-09-19T05:40:00.129056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441086+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:39:12.441086+0000","last_peered":"2024-09-19T05:39:12.441086+0000","last_clean":"2024-09-19T05:39:12.441086+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:39:12.441086+0000","last_undegraded":"2024-09-19T05:39:12.441086+0000","last_fullsized":"2024-09-19T05:39:12.441086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438418+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:39:12.438418+0000","last_peered":"2024-09-19T05:39:12.438418+0000","last_clean":"2024-09-19T05:39:12.438418+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:39:12.438418+0000","last_undegraded":"2024-09-19T05:39:12.438418+0000","last_fullsized":"2024-09-19T05:39:12.438418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441435+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:39:12.441435+0000","last_peered":"2024-09-19T05:39:12.441435+0000","last_clean":"2024-09-19T05:39:12.441435+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:39:12.441435+0000","last_undegraded":"2024-09-19T05:39:12.441435+0000","last_fullsized":"2024-09-19T05:39:12.441435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.402003+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:39:12.402003+0000","last_peered":"2024-09-19T05:39:12.402003+0000","last_clean":"2024-09-19T05:39:12.402003+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:39:12.402003+0000","last_undegraded":"2024-09-19T05:39:12.402003+0000","last_fullsized":"2024-09-19T05:39:12.402003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397355+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:39:12.397355+0000","last_peered":"2024-09-19T05:39:12.397355+0000","last_clean":"2024-09-19T05:39:12.397355+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:39:12.397355+0000","last_undegraded":"2024-09-19T05:39:12.397355+0000","last_fullsized":"2024-09-19T05:39:12.397355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.103444+0000","last_change":"2024-09-19T05:38:59.855714+0000","last_active":"2024-09-19T05:39:48.103444+0000","last_peered":"2024-09-19T05:39:48.103444+0000","last_clean":"2024-09-19T05:39:48.103444+0000","last_became_active":"2024-09-19T05:38:59.855268+0000","last_became_peered":"2024-09-19T05:38:59.855268+0000","last_unstale":"2024-09-19T05:39:48.103444+0000","last_undegraded":"2024-09-19T05:39:48.103444+0000","last_fullsized":"2024-09-19T05:39:48.103444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:20.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440854+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:39:12.440854+0000","last_peered":"2024-09-19T05:39:12.440854+0000","last_clean":"2024-09-19T05:39:12.440854+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:39:12.440854+0000","last_undegraded":"2024-09-19T05:39:12.440854+0000","last_fullsized":"2024-09-19T05:39:12.440854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397855+0000","last_change":"2024-09-19T05:39:08.950477+0000","last_active":"2024-09-19T05:39:12.397855+0000","last_peered":"2024-09-19T05:39:12.397855+0000","last_clean":"2024-09-19T05:39:12.397855+0000","last_became_active":"2024-09-19T05:39:08.949207+0000","last_became_peered":"2024-09-19T05:39:08.949207+0000","last_unstale":"2024-09-19T05:39:12.397855+0000","last_undegraded":"2024-09-19T05:39:12.397855+0000","last_fullsized":"2024-09-19T05:39:12.397855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:05.264531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108952+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:39:47.108952+0000","last_peered":"2024-09-19T05:39:47.108952+0000","last_clean":"2024-09-19T05:39:47.108952+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:39:47.108952+0000","last_undegraded":"2024-09-19T05:39:47.108952+0000","last_fullsized":"2024-09-19T05:39:47.108952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440354+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:39:12.440354+0000","last_peered":"2024-09-19T05:39:12.440354+0000","last_clean":"2024-09-19T05:39:12.440354+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:39:12.440354+0000","last_undegraded":"2024-09-19T05:39:12.440354+0000","last_fullsized":"2024-09-19T05:39:12.440354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397413+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:39:12.397413+0000","last_peered":"2024-09-19T05:39:12.397413+0000","last_clean":"2024-09-19T05:39:12.397413+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:39:12.397413+0000","last_undegraded":"2024-09-19T05:39:12.397413+0000","last_fullsized":"2024-09-19T05:39:12.397413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440338+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:39:12.440338+0000","last_peered":"2024-09-19T05:39:12.440338+0000","last_clean":"2024-09-19T05:39:12.440338+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:39:12.440338+0000","last_undegraded":"2024-09-19T05:39:12.440338+0000","last_fullsized":"2024-09-19T05:39:12.440338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:44.082112+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:39:44.082112+0000","last_peered":"2024-09-19T05:39:44.082112+0000","last_clean":"2024-09-19T05:39:44.082112+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:39:44.082112+0000","last_undegraded":"2024-09-19T05:39:44.082112+0000","last_fullsized":"2024-09-19T05:39:44.082112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397863+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:39:12.397863+0000","last_peered":"2024-09-19T05:39:12.397863+0000","last_clean":"2024-09-19T05:39:12.397863+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:39:12.397863+0000","last_undegraded":"2024-09-19T05:39:12.397863+0000","last_fullsized":"2024-09-19T05:39:12.397863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440557+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:39:12.440557+0000","last_peered":"2024-09-19T05:39:12.440557+0000","last_clean":"2024-09-19T05:39:12.440557+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:39:12.440557+0000","last_undegraded":"2024-09-19T05:39:12.440557+0000","last_fullsized":"2024-09-19T05:39:12.440557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.101040+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:39:46.101040+0000","last_peered":"2024-09-19T05:39:46.101040+0000","last_clean":"2024-09-19T05:39:46.101040+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:39:46.101040+0000","last_undegraded":"2024-09-19T05:39:46.101040+0000","last_fullsized":"2024-09-19T05:39:46.101040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397919+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:39:12.397919+0000","last_peered":"2024-09-19T05:39:12.397919+0000","last_clean":"2024-09-19T05:39:12.397919+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:39:12.397919+0000","last_undegraded":"2024-09-19T05:39:12.397919+0000","last_fullsized":"2024-09-19T05:39:12.397919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439948+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:39:12.439948+0000","last_peered":"2024-09-19T05:39:12.439948+0000","last_clean":"2024-09-19T05:39:12.439948+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:39:12.439948+0000","last_undegraded":"2024-09-19T05:39:12.439948+0000","last_fullsized":"2024-09-19T05:39:12.439948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107481+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:39:47.107481+0000","last_peered":"2024-09-19T05:39:47.107481+0000","last_clean":"2024-09-19T05:39:47.107481+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:39:47.107481+0000","last_undegraded":"2024-09-19T05:39:47.107481+0000","last_fullsized":"2024-09-19T05:39:47.107481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441678+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:39:12.441678+0000","last_peered":"2024-09-19T05:39:12.441678+0000","last_clean":"2024-09-19T05:39:12.441678+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:39:12.441678+0000","last_undegraded":"2024-09-19T05:39:12.441678+0000","last_fullsized":"2024-09-19T05:39:12.441678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397083+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:39:12.397083+0000","last_peered":"2024-09-19T05:39:12.397083+0000","last_clean":"2024-09-19T05:39:12.397083+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:39:12.397083+0000","last_undegraded":"2024-09-19T05:39:12.397083+0000","last_fullsized":"2024-09-19T05:39:12.397083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:44.084264+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:39:44.084264+0000","last_peered":"2024-09-19T05:39:44.084264+0000","last_clean":"2024-09-19T05:39:44.084264+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:39:44.084264+0000","last_undegraded":"2024-09-19T05:39:44.084264+0000","last_fullsized":"2024-09-19T05:39:44.084264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.398380+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:39:12.398380+0000","last_peered":"2024-09-19T05:39:12.398380+0000","last_clean":"2024-09-19T05:39:12.398380+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:39:12.398380+0000","last_undegraded":"2024-09-19T05:39:12.398380+0000","last_fullsized":"2024-09-19T05:39:12.398380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401772+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:39:12.401772+0000","last_peered":"2024-09-19T05:39:12.401772+0000","last_clean":"2024-09-19T05:39:12.401772+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:39:12.401772+0000","last_undegraded":"2024-09-19T05:39:12.401772+0000","last_fullsized":"2024-09-19T05:39:12.401772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108667+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:39:47.108667+0000","last_peered":"2024-09-19T05:39:47.108667+0000","last_clean":"2024-09-19T05:39:47.108667+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:39:47.108667+0000","last_undegraded":"2024-09-19T05:39:47.108667+0000","last_fullsized":"2024-09-19T05:39:47.108667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438538+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:39:12.438538+0000","last_peered":"2024-09-19T05:39:12.438538+0000","last_clean":"2024-09-19T05:39:12.438538+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:39:12.438538+0000","last_undegraded":"2024-09-19T05:39:12.438538+0000","last_fullsized":"2024-09-19T05:39:12.438538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441276+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:39:12.441276+0000","last_peered":"2024-09-19T05:39:12.441276+0000","last_clean":"2024-09-19T05:39:12.441276+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:39:12.441276+0000","last_undegraded":"2024-09-19T05:39:12.441276+0000","last_fullsized":"2024-09-19T05:39:12.441276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'14","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.914447+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:39:58.914447+0000","last_peered":"2024-09-19T05:39:58.914447+0000","last_clean":"2024-09-19T05:39:58.914447+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:39:58.914447+0000","last_undegraded":"2024-09-19T05:39:58.914447+0000","last_fullsized":"2024-09-19T05:39:58.914447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439024+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:39:12.439024+0000","last_peered":"2024-09-19T05:39:12.439024+0000","last_clean":"2024-09-19T05:39:12.439024+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:39:12.439024+0000","last_undegraded":"2024-09-19T05:39:12.439024+0000","last_fullsized":"2024-09-19T05:39:12.439024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440264+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:39:12.440264+0000","last_peered":"2024-09-19T05:39:12.440264+0000","last_clean":"2024-09-19T05:39:12.440264+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:39:12.440264+0000","last_undegraded":"2024-09-19T05:39:12.440264+0000","last_fullsized":"2024-09-19T05:39:12.440264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.109263+0000","last_change":"2024-09-19T05:39:00.095972+0000","last_active":"2024-09-19T05:39:47.109263+0000","last_peered":"2024-09-19T05:39:47.109263+0000","last_clean":"2024-09-19T05:39:47.109263+0000","last_became_active":"2024-09-19T05:39:00.095558+0000","last_became_peered":"2024-09-19T05:39:00.095558+0000","last_unstale":"2024-09-19T05:39:47.109263+0000","last_undegraded":"2024-09-19T05:39:47.109263+0000","last_fullsized":"2024-09-19T05:39:47.109263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:13.708685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440317+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:39:12.440317+0000","last_peered":"2024-09-19T05:39:12.440317+0000","last_clean":"2024-09-19T05:39:12.440317+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:39:12.440317+0000","last_undegraded":"2024-09-19T05:39:12.440317+0000","last_fullsized":"2024-09-19T05:39:12.440317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440068+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:39:12.440068+0000","last_peered":"2024-09-19T05:39:12.440068+0000","last_clean":"2024-09-19T05:39:12.440068+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:39:12.440068+0000","last_undegraded":"2024-09-19T05:39:12.440068+0000","last_fullsized":"2024-09-19T05:39:12.440068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441279+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:39:12.441279+0000","last_peered":"2024-09-19T05:39:12.441279+0000","last_clean":"2024-09-19T05:39:12.441279+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:39:12.441279+0000","last_undegraded":"2024-09-19T05:39:12.441279+0000","last_fullsized":"2024-09-19T05:39:12.441279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397226+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:39:12.397226+0000","last_peered":"2024-09-19T05:39:12.397226+0000","last_clean":"2024-09-19T05:39:12.397226+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:39:12.397226+0000","last_undegraded":"2024-09-19T05:39:12.397226+0000","last_fullsized":"2024-09-19T05:39:12.397226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440180+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:39:12.440180+0000","last_peered":"2024-09-19T05:39:12.440180+0000","last_clean":"2024-09-19T05:39:12.440180+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:39:12.440180+0000","last_undegraded":"2024-09-19T05:39:12.440180+0000","last_fullsized":"2024-09-19T05:39:12.440180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.126961+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:39:58.126961+0000","last_peered":"2024-09-19T05:39:58.126961+0000","last_clean":"2024-09-19T05:39:58.126961+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:39:58.126961+0000","last_undegraded":"2024-09-19T05:39:58.126961+0000","last_fullsized":"2024-09-19T05:39:58.126961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440114+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:39:12.440114+0000","last_peered":"2024-09-19T05:39:12.440114+0000","last_clean":"2024-09-19T05:39:12.440114+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:39:12.440114+0000","last_undegraded":"2024-09-19T05:39:12.440114+0000","last_fullsized":"2024-09-19T05:39:12.440114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439380+0000","last_change":"2024-09-19T05:39:01.893633+0000","last_active":"2024-09-19T05:39:12.439380+0000","last_peered":"2024-09-19T05:39:12.439380+0000","last_clean":"2024-09-19T05:39:12.439380+0000","last_became_active":"2024-09-19T05:39:01.893308+0000","last_became_peered":"2024-09-19T05:39:01.893308+0000","last_unstale":"2024-09-19T05:39:12.439380+0000","last_undegraded":"2024-09-19T05:39:12.439380+0000","last_fullsized":"2024-09-19T05:39:12.439380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:55.117485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.126534+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:39:58.126534+0000","last_peered":"2024-09-19T05:39:58.126534+0000","last_clean":"2024-09-19T05:39:58.126534+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:39:58.126534+0000","last_undegraded":"2024-09-19T05:39:58.126534+0000","last_fullsized":"2024-09-19T05:39:58.126534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440164+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:39:12.440164+0000","last_peered":"2024-09-19T05:39:12.440164+0000","last_clean":"2024-09-19T05:39:12.440164+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:39:12.440164+0000","last_undegraded":"2024-09-19T05:39:12.440164+0000","last_fullsized":"2024-09-19T05:39:12.440164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439336+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:39:12.439336+0000","last_peered":"2024-09-19T05:39:12.439336+0000","last_clean":"2024-09-19T05:39:12.439336+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:39:12.439336+0000","last_undegraded":"2024-09-19T05:39:12.439336+0000","last_fullsized":"2024-09-19T05:39:12.439336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:45.091867+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:39:45.091867+0000","last_peered":"2024-09-19T05:39:45.091867+0000","last_clean":"2024-09-19T05:39:45.091867+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:39:45.091867+0000","last_undegraded":"2024-09-19T05:39:45.091867+0000","last_fullsized":"2024-09-19T05:39:45.091867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439569+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:39:12.439569+0000","last_peered":"2024-09-19T05:39:12.439569+0000","last_clean":"2024-09-19T05:39:12.439569+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:39:12.439569+0000","last_undegraded":"2024-09-19T05:39:12.439569+0000","last_fullsized":"2024-09-19T05:39:12.439569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440707+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:39:12.440707+0000","last_peered":"2024-09-19T05:39:12.440707+0000","last_clean":"2024-09-19T05:39:12.440707+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:39:12.440707+0000","last_undegraded":"2024-09-19T05:39:12.440707+0000","last_fullsized":"2024-09-19T05:39:12.440707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.109656+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:39:48.109656+0000","last_peered":"2024-09-19T05:39:48.109656+0000","last_clean":"2024-09-19T05:39:48.109656+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:39:48.109656+0000","last_undegraded":"2024-09-19T05:39:48.109656+0000","last_fullsized":"2024-09-19T05:39:48.109656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440062+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:39:12.440062+0000","last_peered":"2024-09-19T05:39:12.440062+0000","last_clean":"2024-09-19T05:39:12.440062+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:39:12.440062+0000","last_undegraded":"2024-09-19T05:39:12.440062+0000","last_fullsized":"2024-09-19T05:39:12.440062+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440553+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:39:12.440553+0000","last_peered":"2024-09-19T05:39:12.440553+0000","last_clean":"2024-09-19T05:39:12.440553+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:39:12.440553+0000","last_undegraded":"2024-09-19T05:39:12.440553+0000","last_fullsized":"2024-09-19T05:39:12.440553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440644+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:39:12.440644+0000","last_peered":"2024-09-19T05:39:12.440644+0000","last_clean":"2024-09-19T05:39:12.440644+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:39:12.440644+0000","last_undegraded":"2024-09-19T05:39:12.440644+0000","last_fullsized":"2024-09-19T05:39:12.440644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:57.126561+0000","last_change":"2024-09-19T05:39:00.097011+0000","last_active":"2024-09-19T05:39:57.126561+0000","last_peered":"2024-09-19T05:39:57.126561+0000","last_clean":"2024-09-19T05:39:57.126561+0000","last_became_active":"2024-09-19T05:39:00.096349+0000","last_became_peered":"2024-09-19T05:39:00.096349+0000","last_unstale":"2024-09-19T05:39:57.126561+0000","last_undegraded":"2024-09-19T05:39:57.126561+0000","last_fullsized":"2024-09-19T05:39:57.126561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:45.484368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438850+0000","last_change":"2024-09-19T05:39:08.951614+0000","last_active":"2024-09-19T05:39:12.438850+0000","last_peered":"2024-09-19T05:39:12.438850+0000","last_clean":"2024-09-19T05:39:12.438850+0000","last_became_active":"2024-09-19T05:39:08.951164+0000","last_became_peered":"2024-09-19T05:39:08.951164+0000","last_unstale":"2024-09-19T05:39:12.438850+0000","last_undegraded":"2024-09-19T05:39:12.438850+0000","last_fullsized":"2024-09-19T05:39:12.438850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:47.180400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439825+0000","last_change":"2024-09-19T05:39:01.883269+0000","last_active":"2024-09-19T05:39:12.439825+0000","last_peered":"2024-09-19T05:39:12.439825+0000","last_clean":"2024-09-19T05:39:12.439825+0000","last_became_active":"2024-09-19T05:39:01.882923+0000","last_became_peered":"2024-09-19T05:39:01.882923+0000","last_unstale":"2024-09-19T05:39:12.439825+0000","last_undegraded":"2024-09-19T05:39:12.439825+0000","last_fullsized":"2024-09-19T05:39:12.439825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:05.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107641+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:39:47.107641+0000","last_peered":"2024-09-19T05:39:47.107641+0000","last_clean":"2024-09-19T05:39:47.107641+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:39:47.107641+0000","last_undegraded":"2024-09-19T05:39:47.107641+0000","last_fullsized":"2024-09-19T05:39:47.107641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401813+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:39:12.401813+0000","last_peered":"2024-09-19T05:39:12.401813+0000","last_clean":"2024-09-19T05:39:12.401813+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:39:12.401813+0000","last_undegraded":"2024-09-19T05:39:12.401813+0000","last_fullsized":"2024-09-19T05:39:12.401813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439581+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:39:12.439581+0000","last_peered":"2024-09-19T05:39:12.439581+0000","last_clean":"2024-09-19T05:39:12.439581+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:39:12.439581+0000","last_undegraded":"2024-09-19T05:39:12.439581+0000","last_fullsized":"2024-09-19T05:39:12.439581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.105859+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:39:47.105859+0000","last_peered":"2024-09-19T05:39:47.105859+0000","last_clean":"2024-09-19T05:39:47.105859+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:39:47.105859+0000","last_undegraded":"2024-09-19T05:39:47.105859+0000","last_fullsized":"2024-09-19T05:39:47.105859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440099+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:39:12.440099+0000","last_peered":"2024-09-19T05:39:12.440099+0000","last_clean":"2024-09-19T05:39:12.440099+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:39:12.440099+0000","last_undegraded":"2024-09-19T05:39:12.440099+0000","last_fullsized":"2024-09-19T05:39:12.440099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440185+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:39:12.440185+0000","last_peered":"2024-09-19T05:39:12.440185+0000","last_clean":"2024-09-19T05:39:12.440185+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:39:12.440185+0000","last_undegraded":"2024-09-19T05:39:12.440185+0000","last_fullsized":"2024-09-19T05:39:12.440185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.088459+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:39:46.088459+0000","last_peered":"2024-09-19T05:39:46.088459+0000","last_clean":"2024-09-19T05:39:46.088459+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:39:46.088459+0000","last_undegraded":"2024-09-19T05:39:46.088459+0000","last_fullsized":"2024-09-19T05:39:46.088459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439086+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:39:12.439086+0000","last_peered":"2024-09-19T05:39:12.439086+0000","last_clean":"2024-09-19T05:39:12.439086+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:39:12.439086+0000","last_undegraded":"2024-09-19T05:39:12.439086+0000","last_fullsized":"2024-09-19T05:39:12.439086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439763+0000","last_change":"2024-09-19T05:39:01.897657+0000","last_active":"2024-09-19T05:39:12.439763+0000","last_peered":"2024-09-19T05:39:12.439763+0000","last_clean":"2024-09-19T05:39:12.439763+0000","last_became_active":"2024-09-19T05:39:01.897257+0000","last_became_peered":"2024-09-19T05:39:01.897257+0000","last_unstale":"2024-09-19T05:39:12.439763+0000","last_undegraded":"2024-09-19T05:39:12.439763+0000","last_fullsized":"2024-09-19T05:39:12.439763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:15.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107638+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:39:47.107638+0000","last_peered":"2024-09-19T05:39:47.107638+0000","last_clean":"2024-09-19T05:39:47.107638+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:39:47.107638+0000","last_undegraded":"2024-09-19T05:39:47.107638+0000","last_fullsized":"2024-09-19T05:39:47.107638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440012+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:39:12.440012+0000","last_peered":"2024-09-19T05:39:12.440012+0000","last_clean":"2024-09-19T05:39:12.440012+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:39:12.440012+0000","last_undegraded":"2024-09-19T05:39:12.440012+0000","last_fullsized":"2024-09-19T05:39:12.440012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439939+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:39:12.439939+0000","last_peered":"2024-09-19T05:39:12.439939+0000","last_clean":"2024-09-19T05:39:12.439939+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:39:12.439939+0000","last_undegraded":"2024-09-19T05:39:12.439939+0000","last_fullsized":"2024-09-19T05:39:12.439939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:57.125786+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:39:57.125786+0000","last_peered":"2024-09-19T05:39:57.125786+0000","last_clean":"2024-09-19T05:39:57.125786+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:39:57.125786+0000","last_undegraded":"2024-09-19T05:39:57.125786+0000","last_fullsized":"2024-09-19T05:39:57.125786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441750+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:39:12.441750+0000","last_peered":"2024-09-19T05:39:12.441750+0000","last_clean":"2024-09-19T05:39:12.441750+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:39:12.441750+0000","last_undegraded":"2024-09-19T05:39:12.441750+0000","last_fullsized":"2024-09-19T05:39:12.441750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440020+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:39:12.440020+0000","last_peered":"2024-09-19T05:39:12.440020+0000","last_clean":"2024-09-19T05:39:12.440020+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:39:12.440020+0000","last_undegraded":"2024-09-19T05:39:12.440020+0000","last_fullsized":"2024-09-19T05:39:12.440020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108800+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:39:47.108800+0000","last_peered":"2024-09-19T05:39:47.108800+0000","last_clean":"2024-09-19T05:39:47.108800+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:39:47.108800+0000","last_undegraded":"2024-09-19T05:39:47.108800+0000","last_fullsized":"2024-09-19T05:39:47.108800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439764+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:39:12.439764+0000","last_peered":"2024-09-19T05:39:12.439764+0000","last_clean":"2024-09-19T05:39:12.439764+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:39:12.439764+0000","last_undegraded":"2024-09-19T05:39:12.439764+0000","last_fullsized":"2024-09-19T05:39:12.439764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439944+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:39:12.439944+0000","last_peered":"2024-09-19T05:39:12.439944+0000","last_clean":"2024-09-19T05:39:12.439944+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:39:12.439944+0000","last_undegraded":"2024-09-19T05:39:12.439944+0000","last_fullsized":"2024-09-19T05:39:12.439944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.109403+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:39:48.109403+0000","last_peered":"2024-09-19T05:39:48.109403+0000","last_clean":"2024-09-19T05:39:48.109403+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:39:48.109403+0000","last_undegraded":"2024-09-19T05:39:48.109403+0000","last_fullsized":"2024-09-19T05:39:48.109403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440376+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:39:12.440376+0000","last_peered":"2024-09-19T05:39:12.440376+0000","last_clean":"2024-09-19T05:39:12.440376+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:39:12.440376+0000","last_undegraded":"2024-09-19T05:39:12.440376+0000","last_fullsized":"2024-09-19T05:39:12.440376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397550+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:39:12.397550+0000","last_peered":"2024-09-19T05:39:12.397550+0000","last_clean":"2024-09-19T05:39:12.397550+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:39:12.397550+0000","last_undegraded":"2024-09-19T05:39:12.397550+0000","last_fullsized":"2024-09-19T05:39:12.397550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:01.133226+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:40:01.133226+0000","last_peered":"2024-09-19T05:40:01.133226+0000","last_clean":"2024-09-19T05:40:01.133226+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:40:01.133226+0000","last_undegraded":"2024-09-19T05:40:01.133226+0000","last_fullsized":"2024-09-19T05:40:01.133226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51899,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440259+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:39:12.440259+0000","last_peered":"2024-09-19T05:39:12.440259+0000","last_clean":"2024-09-19T05:39:12.440259+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:39:12.440259+0000","last_undegraded":"2024-09-19T05:39:12.440259+0000","last_fullsized":"2024-09-19T05:39:12.440259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440943+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:39:12.440943+0000","last_peered":"2024-09-19T05:39:12.440943+0000","last_clean":"2024-09-19T05:39:12.440943+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:39:12.440943+0000","last_undegraded":"2024-09-19T05:39:12.440943+0000","last_fullsized":"2024-09-19T05:39:12.440943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.099926+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:39:46.099926+0000","last_peered":"2024-09-19T05:39:46.099926+0000","last_clean":"2024-09-19T05:39:46.099926+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:39:46.099926+0000","last_undegraded":"2024-09-19T05:39:46.099926+0000","last_fullsized":"2024-09-19T05:39:46.099926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397738+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:39:12.397738+0000","last_peered":"2024-09-19T05:39:12.397738+0000","last_clean":"2024-09-19T05:39:12.397738+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:39:12.397738+0000","last_undegraded":"2024-09-19T05:39:12.397738+0000","last_fullsized":"2024-09-19T05:39:12.397738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.402027+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:39:12.402027+0000","last_peered":"2024-09-19T05:39:12.402027+0000","last_clean":"2024-09-19T05:39:12.402027+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:39:12.402027+0000","last_undegraded":"2024-09-19T05:39:12.402027+0000","last_fullsized":"2024-09-19T05:39:12.402027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108621+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:39:47.108621+0000","last_peered":"2024-09-19T05:39:47.108621+0000","last_clean":"2024-09-19T05:39:47.108621+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:39:47.108621+0000","last_undegraded":"2024-09-19T05:39:47.108621+0000","last_fullsized":"2024-09-19T05:39:47.108621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439284+0000","last_change":"2024-09-19T05:39:08.955374+0000","last_active":"2024-09-19T05:39:12.439284+0000","last_peered":"2024-09-19T05:39:12.439284+0000","last_clean":"2024-09-19T05:39:12.439284+0000","last_became_active":"2024-09-19T05:39:08.954981+0000","last_became_peered":"2024-09-19T05:39:08.954981+0000","last_unstale":"2024-09-19T05:39:12.439284+0000","last_undegraded":"2024-09-19T05:39:12.439284+0000","last_fullsized":"2024-09-19T05:39:12.439284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:15.278739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440876+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:39:12.440876+0000","last_peered":"2024-09-19T05:39:12.440876+0000","last_clean":"2024-09-19T05:39:12.440876+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:39:12.440876+0000","last_undegraded":"2024-09-19T05:39:12.440876+0000","last_fullsized":"2024-09-19T05:39:12.440876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:01.131161+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:40:01.131161+0000","last_peered":"2024-09-19T05:40:01.131161+0000","last_clean":"2024-09-19T05:40:01.131161+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:40:01.131161+0000","last_undegraded":"2024-09-19T05:40:01.131161+0000","last_fullsized":"2024-09-19T05:40:01.131161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440098+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:39:12.440098+0000","last_peered":"2024-09-19T05:39:12.440098+0000","last_clean":"2024-09-19T05:39:12.440098+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:39:12.440098+0000","last_undegraded":"2024-09-19T05:39:12.440098+0000","last_fullsized":"2024-09-19T05:39:12.440098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401663+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:39:12.401663+0000","last_peered":"2024-09-19T05:39:12.401663+0000","last_clean":"2024-09-19T05:39:12.401663+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:39:12.401663+0000","last_undegraded":"2024-09-19T05:39:12.401663+0000","last_fullsized":"2024-09-19T05:39:12.401663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439792+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:39:12.439792+0000","last_peered":"2024-09-19T05:39:12.439792+0000","last_clean":"2024-09-19T05:39:12.439792+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:39:12.439792+0000","last_undegraded":"2024-09-19T05:39:12.439792+0000","last_fullsized":"2024-09-19T05:39:12.439792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108665+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:39:47.108665+0000","last_peered":"2024-09-19T05:39:47.108665+0000","last_clean":"2024-09-19T05:39:47.108665+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:39:47.108665+0000","last_undegraded":"2024-09-19T05:39:47.108665+0000","last_fullsized":"2024-09-19T05:39:47.108665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440486+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:39:12.440486+0000","last_peered":"2024-09-19T05:39:12.440486+0000","last_clean":"2024-09-19T05:39:12.440486+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:39:12.440486+0000","last_undegraded":"2024-09-19T05:39:12.440486+0000","last_fullsized":"2024-09-19T05:39:12.440486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439412+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:39:12.439412+0000","last_peered":"2024-09-19T05:39:12.439412+0000","last_clean":"2024-09-19T05:39:12.439412+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:39:12.439412+0000","last_undegraded":"2024-09-19T05:39:12.439412+0000","last_fullsized":"2024-09-19T05:39:12.439412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:45.092424+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:39:45.092424+0000","last_peered":"2024-09-19T05:39:45.092424+0000","last_clean":"2024-09-19T05:39:45.092424+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:39:45.092424+0000","last_undegraded":"2024-09-19T05:39:45.092424+0000","last_fullsized":"2024-09-19T05:39:45.092424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.100861+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:39:46.100861+0000","last_peered":"2024-09-19T05:39:46.100861+0000","last_clean":"2024-09-19T05:39:46.100861+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:39:46.100861+0000","last_undegraded":"2024-09-19T05:39:46.100861+0000","last_fullsized":"2024-09-19T05:39:46.100861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439890+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:39:12.439890+0000","last_peered":"2024-09-19T05:39:12.439890+0000","last_clean":"2024-09-19T05:39:12.439890+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:39:12.439890+0000","last_undegraded":"2024-09-19T05:39:12.439890+0000","last_fullsized":"2024-09-19T05:39:12.439890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438945+0000","last_change":"2024-09-19T05:39:08.951924+0000","last_active":"2024-09-19T05:39:12.438945+0000","last_peered":"2024-09-19T05:39:12.438945+0000","last_clean":"2024-09-19T05:39:12.438945+0000","last_became_active":"2024-09-19T05:39:08.951547+0000","last_became_peered":"2024-09-19T05:39:08.951547+0000","last_unstale":"2024-09-19T05:39:12.438945+0000","last_undegraded":"2024-09-19T05:39:12.438945+0000","last_fullsized":"2024-09-19T05:39:12.438945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:26.564308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:56.125383+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:39:56.125383+0000","last_peered":"2024-09-19T05:39:56.125383+0000","last_clean":"2024-09-19T05:39:56.125383+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:39:56.125383+0000","last_undegraded":"2024-09-19T05:39:56.125383+0000","last_fullsized":"2024-09-19T05:39:56.125383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397211+0000","last_change":"2024-09-19T05:39:01.891620+0000","last_active":"2024-09-19T05:39:12.397211+0000","last_peered":"2024-09-19T05:39:12.397211+0000","last_clean":"2024-09-19T05:39:12.397211+0000","last_became_active":"2024-09-19T05:39:01.891268+0000","last_became_peered":"2024-09-19T05:39:01.891268+0000","last_unstale":"2024-09-19T05:39:12.397211+0000","last_undegraded":"2024-09-19T05:39:12.397211+0000","last_fullsized":"2024-09-19T05:39:12.397211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.966387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439236+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:39:12.439236+0000","last_peered":"2024-09-19T05:39:12.439236+0000","last_clean":"2024-09-19T05:39:12.439236+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:39:12.439236+0000","last_undegraded":"2024-09-19T05:39:12.439236+0000","last_fullsized":"2024-09-19T05:39:12.439236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439013+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:39:12.439013+0000","last_peered":"2024-09-19T05:39:12.439013+0000","last_clean":"2024-09-19T05:39:12.439013+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:39:12.439013+0000","last_undegraded":"2024-09-19T05:39:12.439013+0000","last_fullsized":"2024-09-19T05:39:12.439013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438713+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:39:12.438713+0000","last_peered":"2024-09-19T05:39:12.438713+0000","last_clean":"2024-09-19T05:39:12.438713+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:39:12.438713+0000","last_undegraded":"2024-09-19T05:39:12.438713+0000","last_fullsized":"2024-09-19T05:39:12.438713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438586+0000","last_change":"2024-09-19T05:39:01.898618+0000","last_active":"2024-09-19T05:39:12.438586+0000","last_peered":"2024-09-19T05:39:12.438586+0000","last_clean":"2024-09-19T05:39:12.438586+0000","last_became_active":"2024-09-19T05:39:01.898290+0000","last_became_peered":"2024-09-19T05:39:01.898290+0000","last_unstale":"2024-09-19T05:39:12.438586+0000","last_undegraded":"2024-09-19T05:39:12.438586+0000","last_fullsized":"2024-09-19T05:39:12.438586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:19.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401703+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:39:12.401703+0000","last_peered":"2024-09-19T05:39:12.401703+0000","last_clean":"2024-09-19T05:39:12.401703+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:39:12.401703+0000","last_undegraded":"2024-09-19T05:39:12.401703+0000","last_fullsized":"2024-09-19T05:39:12.401703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440225+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:39:12.440225+0000","last_peered":"2024-09-19T05:39:12.440225+0000","last_clean":"2024-09-19T05:39:12.440225+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:39:12.440225+0000","last_undegraded":"2024-09-19T05:39:12.440225+0000","last_fullsized":"2024-09-19T05:39:12.440225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440498+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:39:12.440498+0000","last_peered":"2024-09-19T05:39:12.440498+0000","last_clean":"2024-09-19T05:39:12.440498+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:39:12.440498+0000","last_undegraded":"2024-09-19T05:39:12.440498+0000","last_fullsized":"2024-09-19T05:39:12.440498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439165+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:39:12.439165+0000","last_peered":"2024-09-19T05:39:12.439165+0000","last_clean":"2024-09-19T05:39:12.439165+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:39:12.439165+0000","last_undegraded":"2024-09-19T05:39:12.439165+0000","last_fullsized":"2024-09-19T05:39:12.439165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441183+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:39:12.441183+0000","last_peered":"2024-09-19T05:39:12.441183+0000","last_clean":"2024-09-19T05:39:12.441183+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:39:12.441183+0000","last_undegraded":"2024-09-19T05:39:12.441183+0000","last_fullsized":"2024-09-19T05:39:12.441183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439276+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:39:12.439276+0000","last_peered":"2024-09-19T05:39:12.439276+0000","last_clean":"2024-09-19T05:39:12.439276+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:39:12.439276+0000","last_undegraded":"2024-09-19T05:39:12.439276+0000","last_fullsized":"2024-09-19T05:39:12.439276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440261+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:39:12.440261+0000","last_peered":"2024-09-19T05:39:12.440261+0000","last_clean":"2024-09-19T05:39:12.440261+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:39:12.440261+0000","last_undegraded":"2024-09-19T05:39:12.440261+0000","last_fullsized":"2024-09-19T05:39:12.440261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439177+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:39:12.439177+0000","last_peered":"2024-09-19T05:39:12.439177+0000","last_clean":"2024-09-19T05:39:12.439177+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:39:12.439177+0000","last_undegraded":"2024-09-19T05:39:12.439177+0000","last_fullsized":"2024-09-19T05:39:12.439177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439653+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:39:12.439653+0000","last_peered":"2024-09-19T05:39:12.439653+0000","last_clean":"2024-09-19T05:39:12.439653+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:39:12.439653+0000","last_undegraded":"2024-09-19T05:39:12.439653+0000","last_fullsized":"2024-09-19T05:39:12.439653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439884+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:39:12.439884+0000","last_peered":"2024-09-19T05:39:12.439884+0000","last_clean":"2024-09-19T05:39:12.439884+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:39:12.439884+0000","last_undegraded":"2024-09-19T05:39:12.439884+0000","last_fullsized":"2024-09-19T05:39:12.439884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439672+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:39:12.439672+0000","last_peered":"2024-09-19T05:39:12.439672+0000","last_clean":"2024-09-19T05:39:12.439672+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:39:12.439672+0000","last_undegraded":"2024-09-19T05:39:12.439672+0000","last_fullsized":"2024-09-19T05:39:12.439672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441596+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:39:12.441596+0000","last_peered":"2024-09-19T05:39:12.441596+0000","last_clean":"2024-09-19T05:39:12.441596+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:39:12.441596+0000","last_undegraded":"2024-09-19T05:39:12.441596+0000","last_fullsized":"2024-09-19T05:39:12.441596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440734+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:39:12.440734+0000","last_peered":"2024-09-19T05:39:12.440734+0000","last_clean":"2024-09-19T05:39:12.440734+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:39:12.440734+0000","last_undegraded":"2024-09-19T05:39:12.440734+0000","last_fullsized":"2024-09-19T05:39:12.440734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438680+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:39:12.438680+0000","last_peered":"2024-09-19T05:39:12.438680+0000","last_clean":"2024-09-19T05:39:12.438680+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:39:12.438680+0000","last_undegraded":"2024-09-19T05:39:12.438680+0000","last_fullsized":"2024-09-19T05:39:12.438680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401440+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:39:12.401440+0000","last_peered":"2024-09-19T05:39:12.401440+0000","last_clean":"2024-09-19T05:39:12.401440+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:39:12.401440+0000","last_undegraded":"2024-09-19T05:39:12.401440+0000","last_fullsized":"2024-09-19T05:39:12.401440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401406+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:39:12.401406+0000","last_peered":"2024-09-19T05:39:12.401406+0000","last_clean":"2024-09-19T05:39:12.401406+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:39:12.401406+0000","last_undegraded":"2024-09-19T05:39:12.401406+0000","last_fullsized":"2024-09-19T05:39:12.401406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439574+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:39:12.439574+0000","last_peered":"2024-09-19T05:39:12.439574+0000","last_clean":"2024-09-19T05:39:12.439574+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:39:12.439574+0000","last_undegraded":"2024-09-19T05:39:12.439574+0000","last_fullsized":"2024-09-19T05:39:12.439574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397973+0000","last_change":"2024-09-19T05:39:01.903139+0000","last_active":"2024-09-19T05:39:12.397973+0000","last_peered":"2024-09-19T05:39:12.397973+0000","last_clean":"2024-09-19T05:39:12.397973+0000","last_became_active":"2024-09-19T05:39:01.902831+0000","last_became_peered":"2024-09-19T05:39:01.902831+0000","last_unstale":"2024-09-19T05:39:12.397973+0000","last_undegraded":"2024-09-19T05:39:12.397973+0000","last_fullsized":"2024-09-19T05:39:12.397973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:15.673504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440677+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:39:12.440677+0000","last_peered":"2024-09-19T05:39:12.440677+0000","last_clean":"2024-09-19T05:39:12.440677+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:39:12.440677+0000","last_undegraded":"2024-09-19T05:39:12.440677+0000","last_fullsized":"2024-09-19T05:39:12.440677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440782+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:39:12.440782+0000","last_peered":"2024-09-19T05:39:12.440782+0000","last_clean":"2024-09-19T05:39:12.440782+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:39:12.440782+0000","last_undegraded":"2024-09-19T05:39:12.440782+0000","last_fullsized":"2024-09-19T05:39:12.440782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439361+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:39:12.439361+0000","last_peered":"2024-09-19T05:39:12.439361+0000","last_clean":"2024-09-19T05:39:12.439361+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:39:12.439361+0000","last_undegraded":"2024-09-19T05:39:12.439361+0000","last_fullsized":"2024-09-19T05:39:12.439361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439287+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:39:12.439287+0000","last_peered":"2024-09-19T05:39:12.439287+0000","last_clean":"2024-09-19T05:39:12.439287+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:39:12.439287+0000","last_undegraded":"2024-09-19T05:39:12.439287+0000","last_fullsized":"2024-09-19T05:39:12.439287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439356+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:39:12.439356+0000","last_peered":"2024-09-19T05:39:12.439356+0000","last_clean":"2024-09-19T05:39:12.439356+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:39:12.439356+0000","last_undegraded":"2024-09-19T05:39:12.439356+0000","last_fullsized":"2024-09-19T05:39:12.439356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439935+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:39:12.439935+0000","last_peered":"2024-09-19T05:39:12.439935+0000","last_clean":"2024-09-19T05:39:12.439935+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:39:12.439935+0000","last_undegraded":"2024-09-19T05:39:12.439935+0000","last_fullsized":"2024-09-19T05:39:12.439935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397137+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:39:12.397137+0000","last_peered":"2024-09-19T05:39:12.397137+0000","last_clean":"2024-09-19T05:39:12.397137+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:39:12.397137+0000","last_undegraded":"2024-09-19T05:39:12.397137+0000","last_fullsized":"2024-09-19T05:39:12.397137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397847+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:39:12.397847+0000","last_peered":"2024-09-19T05:39:12.397847+0000","last_clean":"2024-09-19T05:39:12.397847+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:39:12.397847+0000","last_undegraded":"2024-09-19T05:39:12.397847+0000","last_fullsized":"2024-09-19T05:39:12.397847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401761+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:39:12.401761+0000","last_peered":"2024-09-19T05:39:12.401761+0000","last_clean":"2024-09-19T05:39:12.401761+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:39:12.401761+0000","last_undegraded":"2024-09-19T05:39:12.401761+0000","last_fullsized":"2024-09-19T05:39:12.401761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401651+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:39:12.401651+0000","last_peered":"2024-09-19T05:39:12.401651+0000","last_clean":"2024-09-19T05:39:12.401651+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:39:12.401651+0000","last_undegraded":"2024-09-19T05:39:12.401651+0000","last_fullsized":"2024-09-19T05:39:12.401651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440658+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:39:12.440658+0000","last_peered":"2024-09-19T05:39:12.440658+0000","last_clean":"2024-09-19T05:39:12.440658+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:39:12.440658+0000","last_undegraded":"2024-09-19T05:39:12.440658+0000","last_fullsized":"2024-09-19T05:39:12.440658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59893,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":167,"num_read_kb":156,"num_write":245,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":179679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547222,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":480891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1119606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939683,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":484524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":480806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1078091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":476325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1072887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":480399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":475640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":521313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":520700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":477781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:40:03.219 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:40:03.478 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:03.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:03 smithi038 ceph-mon[31227]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 17 op/s 2024-09-19T05:40:03.892 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:03 smithi038 ceph-mon[31227]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:03 smithi179 ceph-mon[35781]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 17 op/s 2024-09-19T05:40:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:03 smithi179 ceph-mon[35781]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:03 smithi047 ceph-mon[35327]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 17 op/s 2024-09-19T05:40:03.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:03 smithi047 ceph-mon[35327]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 -- 172.21.15.38:0/2840146731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0106f10 msgr2=0x7fe7c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/2840146731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0106f10 0x7fe7c010ea90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009900 tx=0x7fe7b802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 -- 172.21.15.38:0/2840146731 shutdown_connections 2024-09-19T05:40:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/2840146731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0106f10 0x7fe7c010ea90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:03.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/2840146731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0102790 0x7fe7c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/2840146731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0101de0 0x7fe7c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 -- 172.21.15.38:0/2840146731 >> 172.21.15.38:0/2840146731 conn(0x7fe7c00fd8c0 msgr2=0x7fe7c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 -- 172.21.15.38:0/2840146731 shutdown_connections 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 -- 172.21.15.38:0/2840146731 wait complete. 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.971+0000 7fe7c56e1640 1 Processor -- start 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 -- start start 2024-09-19T05:40:03.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0101de0 0x7fe7c0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0106f10 0x7fe7c019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0111f70 con 0x7fe7c0102790 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0111df0 con 0x7fe7c0101de0 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7c56e1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01120f0 con 0x7fe7c0106f10 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47980/0 (socket says 172.21.15.38:47980) 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 -- 172.21.15.38:0/1405611454 learned_addr learned my addr 172.21.15.38:0/1405611454 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:03.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7beffd640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0101de0 0x7fe7c0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7bf7fe640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0106f10 0x7fe7c019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 -- 172.21.15.38:0/1405611454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0106f10 msgr2=0x7fe7c019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0106f10 0x7fe7c019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 -- 172.21.15.38:0/1405611454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0101de0 msgr2=0x7fe7c0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0101de0 0x7fe7c0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 -- 172.21.15.38:0/1405611454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c019d300 con 0x7fe7c0102790 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.972+0000 7fe7be7fc640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800c890 tx=0x7fe7a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.973+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004220 con 0x7fe7c0102790 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.973+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c019d5f0 con 0x7fe7c0102790 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.973+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a80043c0 con 0x7fe7c0102790 2024-09-19T05:40:03.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.973+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004a00 con 0x7fe7c0102790 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.973+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a4ed0 con 0x7fe7c0102790 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.974+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c01021c0 con 0x7fe7c0102790 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.975+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe7a80079b0 con 0x7fe7c0102790 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.976+0000 7fe7b7fff640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 0x7fe7ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.976+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fe7a80cfcc0 con 0x7fe7c0102790 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.976+0000 7fe7beffd640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 0x7fe7ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.976+0000 7fe7beffd640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 0x7fe7ac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007ec0 tx=0x7fe7b00075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:03.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:03.979+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7a8014030 con 0x7fe7c0102790 2024-09-19T05:40:04.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.154+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7c0062630 con 0x7fe7ac0a32d0 2024-09-19T05:40:04.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.162+0000 7fe7b7fff640 1 -- 172.21.15.38:0/1405611454 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340379 (secure 0 0 0) 0x7fe7c0062630 con 0x7fe7ac0a32d0 2024-09-19T05:40:04.166 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:04.168 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:40:04.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 msgr2=0x7fe7ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:04.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 0x7fe7ac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007ec0 tx=0x7fe7b00075b0 comp rx=0 tx=0).stop 2024-09-19T05:40:04.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 msgr2=0x7fe7c0198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800c890 tx=0x7fe7a800cd50 comp rx=0 tx=0).stop 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 shutdown_connections 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7ac0a32d0 0x7fe7ac0a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7c0106f10 0x7fe7c019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7c0102790 0x7fe7c0198890 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 --2- 172.21.15.38:0/1405611454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c0101de0 0x7fe7c0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.165+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 >> 172.21.15.38:0/1405611454 conn(0x7fe7c00fd8c0 msgr2=0x7fe7c00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.166+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 shutdown_connections 2024-09-19T05:40:04.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:04.166+0000 7fe7c56e1640 1 -- 172.21.15.38:0/1405611454 wait complete. 2024-09-19T05:40:04.942 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-19T05:40:03.597827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650495,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":308,"num_read_kb":265,"num_write":428,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354556,"kb_used_data":13628,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587268,"statfs":{"total":1151940427776,"available":1151577362432,"internally_reserved":0,"allocated":13955072,"data_stored":7686071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29125,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":103,"num_write":28,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:00.129056+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:40:00.129056+0000","last_peered":"2024-09-19T05:40:00.129056+0000","last_clean":"2024-09-19T05:40:00.129056+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:40:00.129056+0000","last_undegraded":"2024-09-19T05:40:00.129056+0000","last_fullsized":"2024-09-19T05:40:00.129056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441086+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:39:12.441086+0000","last_peered":"2024-09-19T05:39:12.441086+0000","last_clean":"2024-09-19T05:39:12.441086+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:39:12.441086+0000","last_undegraded":"2024-09-19T05:39:12.441086+0000","last_fullsized":"2024-09-19T05:39:12.441086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438418+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:39:12.438418+0000","last_peered":"2024-09-19T05:39:12.438418+0000","last_clean":"2024-09-19T05:39:12.438418+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:39:12.438418+0000","last_undegraded":"2024-09-19T05:39:12.438418+0000","last_fullsized":"2024-09-19T05:39:12.438418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441435+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:39:12.441435+0000","last_peered":"2024-09-19T05:39:12.441435+0000","last_clean":"2024-09-19T05:39:12.441435+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:39:12.441435+0000","last_undegraded":"2024-09-19T05:39:12.441435+0000","last_fullsized":"2024-09-19T05:39:12.441435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.402003+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:39:12.402003+0000","last_peered":"2024-09-19T05:39:12.402003+0000","last_clean":"2024-09-19T05:39:12.402003+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:39:12.402003+0000","last_undegraded":"2024-09-19T05:39:12.402003+0000","last_fullsized":"2024-09-19T05:39:12.402003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397355+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:39:12.397355+0000","last_peered":"2024-09-19T05:39:12.397355+0000","last_clean":"2024-09-19T05:39:12.397355+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:39:12.397355+0000","last_undegraded":"2024-09-19T05:39:12.397355+0000","last_fullsized":"2024-09-19T05:39:12.397355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.103444+0000","last_change":"2024-09-19T05:38:59.855714+0000","last_active":"2024-09-19T05:39:48.103444+0000","last_peered":"2024-09-19T05:39:48.103444+0000","last_clean":"2024-09-19T05:39:48.103444+0000","last_became_active":"2024-09-19T05:38:59.855268+0000","last_became_peered":"2024-09-19T05:38:59.855268+0000","last_unstale":"2024-09-19T05:39:48.103444+0000","last_undegraded":"2024-09-19T05:39:48.103444+0000","last_fullsized":"2024-09-19T05:39:48.103444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:20.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440854+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:39:12.440854+0000","last_peered":"2024-09-19T05:39:12.440854+0000","last_clean":"2024-09-19T05:39:12.440854+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:39:12.440854+0000","last_undegraded":"2024-09-19T05:39:12.440854+0000","last_fullsized":"2024-09-19T05:39:12.440854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397855+0000","last_change":"2024-09-19T05:39:08.950477+0000","last_active":"2024-09-19T05:39:12.397855+0000","last_peered":"2024-09-19T05:39:12.397855+0000","last_clean":"2024-09-19T05:39:12.397855+0000","last_became_active":"2024-09-19T05:39:08.949207+0000","last_became_peered":"2024-09-19T05:39:08.949207+0000","last_unstale":"2024-09-19T05:39:12.397855+0000","last_undegraded":"2024-09-19T05:39:12.397855+0000","last_fullsized":"2024-09-19T05:39:12.397855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:05.264531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108952+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:39:47.108952+0000","last_peered":"2024-09-19T05:39:47.108952+0000","last_clean":"2024-09-19T05:39:47.108952+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:39:47.108952+0000","last_undegraded":"2024-09-19T05:39:47.108952+0000","last_fullsized":"2024-09-19T05:39:47.108952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440354+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:39:12.440354+0000","last_peered":"2024-09-19T05:39:12.440354+0000","last_clean":"2024-09-19T05:39:12.440354+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:39:12.440354+0000","last_undegraded":"2024-09-19T05:39:12.440354+0000","last_fullsized":"2024-09-19T05:39:12.440354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397413+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:39:12.397413+0000","last_peered":"2024-09-19T05:39:12.397413+0000","last_clean":"2024-09-19T05:39:12.397413+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:39:12.397413+0000","last_undegraded":"2024-09-19T05:39:12.397413+0000","last_fullsized":"2024-09-19T05:39:12.397413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440338+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:39:12.440338+0000","last_peered":"2024-09-19T05:39:12.440338+0000","last_clean":"2024-09-19T05:39:12.440338+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:39:12.440338+0000","last_undegraded":"2024-09-19T05:39:12.440338+0000","last_fullsized":"2024-09-19T05:39:12.440338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:44.082112+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:39:44.082112+0000","last_peered":"2024-09-19T05:39:44.082112+0000","last_clean":"2024-09-19T05:39:44.082112+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:39:44.082112+0000","last_undegraded":"2024-09-19T05:39:44.082112+0000","last_fullsized":"2024-09-19T05:39:44.082112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397863+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:39:12.397863+0000","last_peered":"2024-09-19T05:39:12.397863+0000","last_clean":"2024-09-19T05:39:12.397863+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:39:12.397863+0000","last_undegraded":"2024-09-19T05:39:12.397863+0000","last_fullsized":"2024-09-19T05:39:12.397863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440557+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:39:12.440557+0000","last_peered":"2024-09-19T05:39:12.440557+0000","last_clean":"2024-09-19T05:39:12.440557+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:39:12.440557+0000","last_undegraded":"2024-09-19T05:39:12.440557+0000","last_fullsized":"2024-09-19T05:39:12.440557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.101040+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:39:46.101040+0000","last_peered":"2024-09-19T05:39:46.101040+0000","last_clean":"2024-09-19T05:39:46.101040+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:39:46.101040+0000","last_undegraded":"2024-09-19T05:39:46.101040+0000","last_fullsized":"2024-09-19T05:39:46.101040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397919+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:39:12.397919+0000","last_peered":"2024-09-19T05:39:12.397919+0000","last_clean":"2024-09-19T05:39:12.397919+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:39:12.397919+0000","last_undegraded":"2024-09-19T05:39:12.397919+0000","last_fullsized":"2024-09-19T05:39:12.397919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439948+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:39:12.439948+0000","last_peered":"2024-09-19T05:39:12.439948+0000","last_clean":"2024-09-19T05:39:12.439948+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:39:12.439948+0000","last_undegraded":"2024-09-19T05:39:12.439948+0000","last_fullsized":"2024-09-19T05:39:12.439948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107481+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:39:47.107481+0000","last_peered":"2024-09-19T05:39:47.107481+0000","last_clean":"2024-09-19T05:39:47.107481+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:39:47.107481+0000","last_undegraded":"2024-09-19T05:39:47.107481+0000","last_fullsized":"2024-09-19T05:39:47.107481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441678+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:39:12.441678+0000","last_peered":"2024-09-19T05:39:12.441678+0000","last_clean":"2024-09-19T05:39:12.441678+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:39:12.441678+0000","last_undegraded":"2024-09-19T05:39:12.441678+0000","last_fullsized":"2024-09-19T05:39:12.441678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397083+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:39:12.397083+0000","last_peered":"2024-09-19T05:39:12.397083+0000","last_clean":"2024-09-19T05:39:12.397083+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:39:12.397083+0000","last_undegraded":"2024-09-19T05:39:12.397083+0000","last_fullsized":"2024-09-19T05:39:12.397083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:44.084264+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:39:44.084264+0000","last_peered":"2024-09-19T05:39:44.084264+0000","last_clean":"2024-09-19T05:39:44.084264+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:39:44.084264+0000","last_undegraded":"2024-09-19T05:39:44.084264+0000","last_fullsized":"2024-09-19T05:39:44.084264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.398380+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:39:12.398380+0000","last_peered":"2024-09-19T05:39:12.398380+0000","last_clean":"2024-09-19T05:39:12.398380+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:39:12.398380+0000","last_undegraded":"2024-09-19T05:39:12.398380+0000","last_fullsized":"2024-09-19T05:39:12.398380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401772+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:39:12.401772+0000","last_peered":"2024-09-19T05:39:12.401772+0000","last_clean":"2024-09-19T05:39:12.401772+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:39:12.401772+0000","last_undegraded":"2024-09-19T05:39:12.401772+0000","last_fullsized":"2024-09-19T05:39:12.401772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108667+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:39:47.108667+0000","last_peered":"2024-09-19T05:39:47.108667+0000","last_clean":"2024-09-19T05:39:47.108667+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:39:47.108667+0000","last_undegraded":"2024-09-19T05:39:47.108667+0000","last_fullsized":"2024-09-19T05:39:47.108667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438538+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:39:12.438538+0000","last_peered":"2024-09-19T05:39:12.438538+0000","last_clean":"2024-09-19T05:39:12.438538+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:39:12.438538+0000","last_undegraded":"2024-09-19T05:39:12.438538+0000","last_fullsized":"2024-09-19T05:39:12.438538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441276+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:39:12.441276+0000","last_peered":"2024-09-19T05:39:12.441276+0000","last_clean":"2024-09-19T05:39:12.441276+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:39:12.441276+0000","last_undegraded":"2024-09-19T05:39:12.441276+0000","last_fullsized":"2024-09-19T05:39:12.441276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'14","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.914447+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:39:58.914447+0000","last_peered":"2024-09-19T05:39:58.914447+0000","last_clean":"2024-09-19T05:39:58.914447+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:39:58.914447+0000","last_undegraded":"2024-09-19T05:39:58.914447+0000","last_fullsized":"2024-09-19T05:39:58.914447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439024+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:39:12.439024+0000","last_peered":"2024-09-19T05:39:12.439024+0000","last_clean":"2024-09-19T05:39:12.439024+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:39:12.439024+0000","last_undegraded":"2024-09-19T05:39:12.439024+0000","last_fullsized":"2024-09-19T05:39:12.439024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440264+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:39:12.440264+0000","last_peered":"2024-09-19T05:39:12.440264+0000","last_clean":"2024-09-19T05:39:12.440264+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:39:12.440264+0000","last_undegraded":"2024-09-19T05:39:12.440264+0000","last_fullsized":"2024-09-19T05:39:12.440264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.109263+0000","last_change":"2024-09-19T05:39:00.095972+0000","last_active":"2024-09-19T05:39:47.109263+0000","last_peered":"2024-09-19T05:39:47.109263+0000","last_clean":"2024-09-19T05:39:47.109263+0000","last_became_active":"2024-09-19T05:39:00.095558+0000","last_became_peered":"2024-09-19T05:39:00.095558+0000","last_unstale":"2024-09-19T05:39:47.109263+0000","last_undegraded":"2024-09-19T05:39:47.109263+0000","last_fullsized":"2024-09-19T05:39:47.109263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:13.708685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440317+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:39:12.440317+0000","last_peered":"2024-09-19T05:39:12.440317+0000","last_clean":"2024-09-19T05:39:12.440317+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:39:12.440317+0000","last_undegraded":"2024-09-19T05:39:12.440317+0000","last_fullsized":"2024-09-19T05:39:12.440317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440068+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:39:12.440068+0000","last_peered":"2024-09-19T05:39:12.440068+0000","last_clean":"2024-09-19T05:39:12.440068+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:39:12.440068+0000","last_undegraded":"2024-09-19T05:39:12.440068+0000","last_fullsized":"2024-09-19T05:39:12.440068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441279+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:39:12.441279+0000","last_peered":"2024-09-19T05:39:12.441279+0000","last_clean":"2024-09-19T05:39:12.441279+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:39:12.441279+0000","last_undegraded":"2024-09-19T05:39:12.441279+0000","last_fullsized":"2024-09-19T05:39:12.441279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397226+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:39:12.397226+0000","last_peered":"2024-09-19T05:39:12.397226+0000","last_clean":"2024-09-19T05:39:12.397226+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:39:12.397226+0000","last_undegraded":"2024-09-19T05:39:12.397226+0000","last_fullsized":"2024-09-19T05:39:12.397226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440180+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:39:12.440180+0000","last_peered":"2024-09-19T05:39:12.440180+0000","last_clean":"2024-09-19T05:39:12.440180+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:39:12.440180+0000","last_undegraded":"2024-09-19T05:39:12.440180+0000","last_fullsized":"2024-09-19T05:39:12.440180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.126961+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:39:58.126961+0000","last_peered":"2024-09-19T05:39:58.126961+0000","last_clean":"2024-09-19T05:39:58.126961+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:39:58.126961+0000","last_undegraded":"2024-09-19T05:39:58.126961+0000","last_fullsized":"2024-09-19T05:39:58.126961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440114+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:39:12.440114+0000","last_peered":"2024-09-19T05:39:12.440114+0000","last_clean":"2024-09-19T05:39:12.440114+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:39:12.440114+0000","last_undegraded":"2024-09-19T05:39:12.440114+0000","last_fullsized":"2024-09-19T05:39:12.440114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439380+0000","last_change":"2024-09-19T05:39:01.893633+0000","last_active":"2024-09-19T05:39:12.439380+0000","last_peered":"2024-09-19T05:39:12.439380+0000","last_clean":"2024-09-19T05:39:12.439380+0000","last_became_active":"2024-09-19T05:39:01.893308+0000","last_became_peered":"2024-09-19T05:39:01.893308+0000","last_unstale":"2024-09-19T05:39:12.439380+0000","last_undegraded":"2024-09-19T05:39:12.439380+0000","last_fullsized":"2024-09-19T05:39:12.439380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:55.117485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:58.126534+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:39:58.126534+0000","last_peered":"2024-09-19T05:39:58.126534+0000","last_clean":"2024-09-19T05:39:58.126534+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:39:58.126534+0000","last_undegraded":"2024-09-19T05:39:58.126534+0000","last_fullsized":"2024-09-19T05:39:58.126534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440164+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:39:12.440164+0000","last_peered":"2024-09-19T05:39:12.440164+0000","last_clean":"2024-09-19T05:39:12.440164+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:39:12.440164+0000","last_undegraded":"2024-09-19T05:39:12.440164+0000","last_fullsized":"2024-09-19T05:39:12.440164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439336+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:39:12.439336+0000","last_peered":"2024-09-19T05:39:12.439336+0000","last_clean":"2024-09-19T05:39:12.439336+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:39:12.439336+0000","last_undegraded":"2024-09-19T05:39:12.439336+0000","last_fullsized":"2024-09-19T05:39:12.439336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:45.091867+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:39:45.091867+0000","last_peered":"2024-09-19T05:39:45.091867+0000","last_clean":"2024-09-19T05:39:45.091867+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:39:45.091867+0000","last_undegraded":"2024-09-19T05:39:45.091867+0000","last_fullsized":"2024-09-19T05:39:45.091867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439569+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:39:12.439569+0000","last_peered":"2024-09-19T05:39:12.439569+0000","last_clean":"2024-09-19T05:39:12.439569+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:39:12.439569+0000","last_undegraded":"2024-09-19T05:39:12.439569+0000","last_fullsized":"2024-09-19T05:39:12.439569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440707+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:39:12.440707+0000","last_peered":"2024-09-19T05:39:12.440707+0000","last_clean":"2024-09-19T05:39:12.440707+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:39:12.440707+0000","last_undegraded":"2024-09-19T05:39:12.440707+0000","last_fullsized":"2024-09-19T05:39:12.440707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.109656+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:39:48.109656+0000","last_peered":"2024-09-19T05:39:48.109656+0000","last_clean":"2024-09-19T05:39:48.109656+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:39:48.109656+0000","last_undegraded":"2024-09-19T05:39:48.109656+0000","last_fullsized":"2024-09-19T05:39:48.109656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440062+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:39:12.440062+0000","last_peered":"2024-09-19T05:39:12.440062+0000","last_clean":"2024-09-19T05:39:12.440062+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:39:12.440062+0000","last_undegraded":"2024-09-19T05:39:12.440062+0000","last_fullsized":"2024-09-19T05:39:12.440062+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440553+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:39:12.440553+0000","last_peered":"2024-09-19T05:39:12.440553+0000","last_clean":"2024-09-19T05:39:12.440553+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:39:12.440553+0000","last_undegraded":"2024-09-19T05:39:12.440553+0000","last_fullsized":"2024-09-19T05:39:12.440553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440644+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:39:12.440644+0000","last_peered":"2024-09-19T05:39:12.440644+0000","last_clean":"2024-09-19T05:39:12.440644+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:39:12.440644+0000","last_undegraded":"2024-09-19T05:39:12.440644+0000","last_fullsized":"2024-09-19T05:39:12.440644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:02.133136+0000","last_change":"2024-09-19T05:39:00.097011+0000","last_active":"2024-09-19T05:40:02.133136+0000","last_peered":"2024-09-19T05:40:02.133136+0000","last_clean":"2024-09-19T05:40:02.133136+0000","last_became_active":"2024-09-19T05:39:00.096349+0000","last_became_peered":"2024-09-19T05:39:00.096349+0000","last_unstale":"2024-09-19T05:40:02.133136+0000","last_undegraded":"2024-09-19T05:40:02.133136+0000","last_fullsized":"2024-09-19T05:40:02.133136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:45.484368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438850+0000","last_change":"2024-09-19T05:39:08.951614+0000","last_active":"2024-09-19T05:39:12.438850+0000","last_peered":"2024-09-19T05:39:12.438850+0000","last_clean":"2024-09-19T05:39:12.438850+0000","last_became_active":"2024-09-19T05:39:08.951164+0000","last_became_peered":"2024-09-19T05:39:08.951164+0000","last_unstale":"2024-09-19T05:39:12.438850+0000","last_undegraded":"2024-09-19T05:39:12.438850+0000","last_fullsized":"2024-09-19T05:39:12.438850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:47.180400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439825+0000","last_change":"2024-09-19T05:39:01.883269+0000","last_active":"2024-09-19T05:39:12.439825+0000","last_peered":"2024-09-19T05:39:12.439825+0000","last_clean":"2024-09-19T05:39:12.439825+0000","last_became_active":"2024-09-19T05:39:01.882923+0000","last_became_peered":"2024-09-19T05:39:01.882923+0000","last_unstale":"2024-09-19T05:39:12.439825+0000","last_undegraded":"2024-09-19T05:39:12.439825+0000","last_fullsized":"2024-09-19T05:39:12.439825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:06:05.153553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107641+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:39:47.107641+0000","last_peered":"2024-09-19T05:39:47.107641+0000","last_clean":"2024-09-19T05:39:47.107641+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:39:47.107641+0000","last_undegraded":"2024-09-19T05:39:47.107641+0000","last_fullsized":"2024-09-19T05:39:47.107641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401813+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:39:12.401813+0000","last_peered":"2024-09-19T05:39:12.401813+0000","last_clean":"2024-09-19T05:39:12.401813+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:39:12.401813+0000","last_undegraded":"2024-09-19T05:39:12.401813+0000","last_fullsized":"2024-09-19T05:39:12.401813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439581+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:39:12.439581+0000","last_peered":"2024-09-19T05:39:12.439581+0000","last_clean":"2024-09-19T05:39:12.439581+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:39:12.439581+0000","last_undegraded":"2024-09-19T05:39:12.439581+0000","last_fullsized":"2024-09-19T05:39:12.439581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.105859+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:39:47.105859+0000","last_peered":"2024-09-19T05:39:47.105859+0000","last_clean":"2024-09-19T05:39:47.105859+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:39:47.105859+0000","last_undegraded":"2024-09-19T05:39:47.105859+0000","last_fullsized":"2024-09-19T05:39:47.105859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440099+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:39:12.440099+0000","last_peered":"2024-09-19T05:39:12.440099+0000","last_clean":"2024-09-19T05:39:12.440099+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:39:12.440099+0000","last_undegraded":"2024-09-19T05:39:12.440099+0000","last_fullsized":"2024-09-19T05:39:12.440099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440185+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:39:12.440185+0000","last_peered":"2024-09-19T05:39:12.440185+0000","last_clean":"2024-09-19T05:39:12.440185+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:39:12.440185+0000","last_undegraded":"2024-09-19T05:39:12.440185+0000","last_fullsized":"2024-09-19T05:39:12.440185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.088459+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:39:46.088459+0000","last_peered":"2024-09-19T05:39:46.088459+0000","last_clean":"2024-09-19T05:39:46.088459+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:39:46.088459+0000","last_undegraded":"2024-09-19T05:39:46.088459+0000","last_fullsized":"2024-09-19T05:39:46.088459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439086+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:39:12.439086+0000","last_peered":"2024-09-19T05:39:12.439086+0000","last_clean":"2024-09-19T05:39:12.439086+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:39:12.439086+0000","last_undegraded":"2024-09-19T05:39:12.439086+0000","last_fullsized":"2024-09-19T05:39:12.439086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439763+0000","last_change":"2024-09-19T05:39:01.897657+0000","last_active":"2024-09-19T05:39:12.439763+0000","last_peered":"2024-09-19T05:39:12.439763+0000","last_clean":"2024-09-19T05:39:12.439763+0000","last_became_active":"2024-09-19T05:39:01.897257+0000","last_became_peered":"2024-09-19T05:39:01.897257+0000","last_unstale":"2024-09-19T05:39:12.439763+0000","last_undegraded":"2024-09-19T05:39:12.439763+0000","last_fullsized":"2024-09-19T05:39:12.439763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:15.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.107638+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:39:47.107638+0000","last_peered":"2024-09-19T05:39:47.107638+0000","last_clean":"2024-09-19T05:39:47.107638+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:39:47.107638+0000","last_undegraded":"2024-09-19T05:39:47.107638+0000","last_fullsized":"2024-09-19T05:39:47.107638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440012+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:39:12.440012+0000","last_peered":"2024-09-19T05:39:12.440012+0000","last_clean":"2024-09-19T05:39:12.440012+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:39:12.440012+0000","last_undegraded":"2024-09-19T05:39:12.440012+0000","last_fullsized":"2024-09-19T05:39:12.440012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439939+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:39:12.439939+0000","last_peered":"2024-09-19T05:39:12.439939+0000","last_clean":"2024-09-19T05:39:12.439939+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:39:12.439939+0000","last_undegraded":"2024-09-19T05:39:12.439939+0000","last_fullsized":"2024-09-19T05:39:12.439939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:02.132368+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:40:02.132368+0000","last_peered":"2024-09-19T05:40:02.132368+0000","last_clean":"2024-09-19T05:40:02.132368+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:40:02.132368+0000","last_undegraded":"2024-09-19T05:40:02.132368+0000","last_fullsized":"2024-09-19T05:40:02.132368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441750+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:39:12.441750+0000","last_peered":"2024-09-19T05:39:12.441750+0000","last_clean":"2024-09-19T05:39:12.441750+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:39:12.441750+0000","last_undegraded":"2024-09-19T05:39:12.441750+0000","last_fullsized":"2024-09-19T05:39:12.441750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440020+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:39:12.440020+0000","last_peered":"2024-09-19T05:39:12.440020+0000","last_clean":"2024-09-19T05:39:12.440020+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:39:12.440020+0000","last_undegraded":"2024-09-19T05:39:12.440020+0000","last_fullsized":"2024-09-19T05:39:12.440020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108800+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:39:47.108800+0000","last_peered":"2024-09-19T05:39:47.108800+0000","last_clean":"2024-09-19T05:39:47.108800+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:39:47.108800+0000","last_undegraded":"2024-09-19T05:39:47.108800+0000","last_fullsized":"2024-09-19T05:39:47.108800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439764+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:39:12.439764+0000","last_peered":"2024-09-19T05:39:12.439764+0000","last_clean":"2024-09-19T05:39:12.439764+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:39:12.439764+0000","last_undegraded":"2024-09-19T05:39:12.439764+0000","last_fullsized":"2024-09-19T05:39:12.439764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439944+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:39:12.439944+0000","last_peered":"2024-09-19T05:39:12.439944+0000","last_clean":"2024-09-19T05:39:12.439944+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:39:12.439944+0000","last_undegraded":"2024-09-19T05:39:12.439944+0000","last_fullsized":"2024-09-19T05:39:12.439944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:48.109403+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:39:48.109403+0000","last_peered":"2024-09-19T05:39:48.109403+0000","last_clean":"2024-09-19T05:39:48.109403+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:39:48.109403+0000","last_undegraded":"2024-09-19T05:39:48.109403+0000","last_fullsized":"2024-09-19T05:39:48.109403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440376+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:39:12.440376+0000","last_peered":"2024-09-19T05:39:12.440376+0000","last_clean":"2024-09-19T05:39:12.440376+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:39:12.440376+0000","last_undegraded":"2024-09-19T05:39:12.440376+0000","last_fullsized":"2024-09-19T05:39:12.440376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397550+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:39:12.397550+0000","last_peered":"2024-09-19T05:39:12.397550+0000","last_clean":"2024-09-19T05:39:12.397550+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:39:12.397550+0000","last_undegraded":"2024-09-19T05:39:12.397550+0000","last_fullsized":"2024-09-19T05:39:12.397550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:01.133226+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:40:01.133226+0000","last_peered":"2024-09-19T05:40:01.133226+0000","last_clean":"2024-09-19T05:40:01.133226+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:40:01.133226+0000","last_undegraded":"2024-09-19T05:40:01.133226+0000","last_fullsized":"2024-09-19T05:40:01.133226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51899,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440259+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:39:12.440259+0000","last_peered":"2024-09-19T05:39:12.440259+0000","last_clean":"2024-09-19T05:39:12.440259+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:39:12.440259+0000","last_undegraded":"2024-09-19T05:39:12.440259+0000","last_fullsized":"2024-09-19T05:39:12.440259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440943+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:39:12.440943+0000","last_peered":"2024-09-19T05:39:12.440943+0000","last_clean":"2024-09-19T05:39:12.440943+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:39:12.440943+0000","last_undegraded":"2024-09-19T05:39:12.440943+0000","last_fullsized":"2024-09-19T05:39:12.440943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.099926+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:39:46.099926+0000","last_peered":"2024-09-19T05:39:46.099926+0000","last_clean":"2024-09-19T05:39:46.099926+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:39:46.099926+0000","last_undegraded":"2024-09-19T05:39:46.099926+0000","last_fullsized":"2024-09-19T05:39:46.099926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397738+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:39:12.397738+0000","last_peered":"2024-09-19T05:39:12.397738+0000","last_clean":"2024-09-19T05:39:12.397738+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:39:12.397738+0000","last_undegraded":"2024-09-19T05:39:12.397738+0000","last_fullsized":"2024-09-19T05:39:12.397738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.402027+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:39:12.402027+0000","last_peered":"2024-09-19T05:39:12.402027+0000","last_clean":"2024-09-19T05:39:12.402027+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:39:12.402027+0000","last_undegraded":"2024-09-19T05:39:12.402027+0000","last_fullsized":"2024-09-19T05:39:12.402027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108621+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:39:47.108621+0000","last_peered":"2024-09-19T05:39:47.108621+0000","last_clean":"2024-09-19T05:39:47.108621+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:39:47.108621+0000","last_undegraded":"2024-09-19T05:39:47.108621+0000","last_fullsized":"2024-09-19T05:39:47.108621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439284+0000","last_change":"2024-09-19T05:39:08.955374+0000","last_active":"2024-09-19T05:39:12.439284+0000","last_peered":"2024-09-19T05:39:12.439284+0000","last_clean":"2024-09-19T05:39:12.439284+0000","last_became_active":"2024-09-19T05:39:08.954981+0000","last_became_peered":"2024-09-19T05:39:08.954981+0000","last_unstale":"2024-09-19T05:39:12.439284+0000","last_undegraded":"2024-09-19T05:39:12.439284+0000","last_fullsized":"2024-09-19T05:39:12.439284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:15.278739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440876+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:39:12.440876+0000","last_peered":"2024-09-19T05:39:12.440876+0000","last_clean":"2024-09-19T05:39:12.440876+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:39:12.440876+0000","last_undegraded":"2024-09-19T05:39:12.440876+0000","last_fullsized":"2024-09-19T05:39:12.440876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:01.131161+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:40:01.131161+0000","last_peered":"2024-09-19T05:40:01.131161+0000","last_clean":"2024-09-19T05:40:01.131161+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:40:01.131161+0000","last_undegraded":"2024-09-19T05:40:01.131161+0000","last_fullsized":"2024-09-19T05:40:01.131161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440098+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:39:12.440098+0000","last_peered":"2024-09-19T05:39:12.440098+0000","last_clean":"2024-09-19T05:39:12.440098+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:39:12.440098+0000","last_undegraded":"2024-09-19T05:39:12.440098+0000","last_fullsized":"2024-09-19T05:39:12.440098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401663+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:39:12.401663+0000","last_peered":"2024-09-19T05:39:12.401663+0000","last_clean":"2024-09-19T05:39:12.401663+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:39:12.401663+0000","last_undegraded":"2024-09-19T05:39:12.401663+0000","last_fullsized":"2024-09-19T05:39:12.401663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439792+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:39:12.439792+0000","last_peered":"2024-09-19T05:39:12.439792+0000","last_clean":"2024-09-19T05:39:12.439792+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:39:12.439792+0000","last_undegraded":"2024-09-19T05:39:12.439792+0000","last_fullsized":"2024-09-19T05:39:12.439792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:47.108665+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:39:47.108665+0000","last_peered":"2024-09-19T05:39:47.108665+0000","last_clean":"2024-09-19T05:39:47.108665+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:39:47.108665+0000","last_undegraded":"2024-09-19T05:39:47.108665+0000","last_fullsized":"2024-09-19T05:39:47.108665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440486+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:39:12.440486+0000","last_peered":"2024-09-19T05:39:12.440486+0000","last_clean":"2024-09-19T05:39:12.440486+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:39:12.440486+0000","last_undegraded":"2024-09-19T05:39:12.440486+0000","last_fullsized":"2024-09-19T05:39:12.440486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439412+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:39:12.439412+0000","last_peered":"2024-09-19T05:39:12.439412+0000","last_clean":"2024-09-19T05:39:12.439412+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:39:12.439412+0000","last_undegraded":"2024-09-19T05:39:12.439412+0000","last_fullsized":"2024-09-19T05:39:12.439412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:45.092424+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:39:45.092424+0000","last_peered":"2024-09-19T05:39:45.092424+0000","last_clean":"2024-09-19T05:39:45.092424+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:39:45.092424+0000","last_undegraded":"2024-09-19T05:39:45.092424+0000","last_fullsized":"2024-09-19T05:39:45.092424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:46.100861+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:39:46.100861+0000","last_peered":"2024-09-19T05:39:46.100861+0000","last_clean":"2024-09-19T05:39:46.100861+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:39:46.100861+0000","last_undegraded":"2024-09-19T05:39:46.100861+0000","last_fullsized":"2024-09-19T05:39:46.100861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439890+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:39:12.439890+0000","last_peered":"2024-09-19T05:39:12.439890+0000","last_clean":"2024-09-19T05:39:12.439890+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:39:12.439890+0000","last_undegraded":"2024-09-19T05:39:12.439890+0000","last_fullsized":"2024-09-19T05:39:12.439890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438945+0000","last_change":"2024-09-19T05:39:08.951924+0000","last_active":"2024-09-19T05:39:12.438945+0000","last_peered":"2024-09-19T05:39:12.438945+0000","last_clean":"2024-09-19T05:39:12.438945+0000","last_became_active":"2024-09-19T05:39:08.951547+0000","last_became_peered":"2024-09-19T05:39:08.951547+0000","last_unstale":"2024-09-19T05:39:12.438945+0000","last_undegraded":"2024-09-19T05:39:12.438945+0000","last_fullsized":"2024-09-19T05:39:12.438945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:26.564308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:40:01.131845+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:40:01.131845+0000","last_peered":"2024-09-19T05:40:01.131845+0000","last_clean":"2024-09-19T05:40:01.131845+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:40:01.131845+0000","last_undegraded":"2024-09-19T05:40:01.131845+0000","last_fullsized":"2024-09-19T05:40:01.131845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397211+0000","last_change":"2024-09-19T05:39:01.891620+0000","last_active":"2024-09-19T05:39:12.397211+0000","last_peered":"2024-09-19T05:39:12.397211+0000","last_clean":"2024-09-19T05:39:12.397211+0000","last_became_active":"2024-09-19T05:39:01.891268+0000","last_became_peered":"2024-09-19T05:39:01.891268+0000","last_unstale":"2024-09-19T05:39:12.397211+0000","last_undegraded":"2024-09-19T05:39:12.397211+0000","last_fullsized":"2024-09-19T05:39:12.397211+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:15.966387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439236+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:39:12.439236+0000","last_peered":"2024-09-19T05:39:12.439236+0000","last_clean":"2024-09-19T05:39:12.439236+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:39:12.439236+0000","last_undegraded":"2024-09-19T05:39:12.439236+0000","last_fullsized":"2024-09-19T05:39:12.439236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439013+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:39:12.439013+0000","last_peered":"2024-09-19T05:39:12.439013+0000","last_clean":"2024-09-19T05:39:12.439013+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:39:12.439013+0000","last_undegraded":"2024-09-19T05:39:12.439013+0000","last_fullsized":"2024-09-19T05:39:12.439013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438713+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:39:12.438713+0000","last_peered":"2024-09-19T05:39:12.438713+0000","last_clean":"2024-09-19T05:39:12.438713+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:39:12.438713+0000","last_undegraded":"2024-09-19T05:39:12.438713+0000","last_fullsized":"2024-09-19T05:39:12.438713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438586+0000","last_change":"2024-09-19T05:39:01.898618+0000","last_active":"2024-09-19T05:39:12.438586+0000","last_peered":"2024-09-19T05:39:12.438586+0000","last_clean":"2024-09-19T05:39:12.438586+0000","last_became_active":"2024-09-19T05:39:01.898290+0000","last_became_peered":"2024-09-19T05:39:01.898290+0000","last_unstale":"2024-09-19T05:39:12.438586+0000","last_undegraded":"2024-09-19T05:39:12.438586+0000","last_fullsized":"2024-09-19T05:39:12.438586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:19.419024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401703+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:39:12.401703+0000","last_peered":"2024-09-19T05:39:12.401703+0000","last_clean":"2024-09-19T05:39:12.401703+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:39:12.401703+0000","last_undegraded":"2024-09-19T05:39:12.401703+0000","last_fullsized":"2024-09-19T05:39:12.401703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440225+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:39:12.440225+0000","last_peered":"2024-09-19T05:39:12.440225+0000","last_clean":"2024-09-19T05:39:12.440225+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:39:12.440225+0000","last_undegraded":"2024-09-19T05:39:12.440225+0000","last_fullsized":"2024-09-19T05:39:12.440225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440498+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:39:12.440498+0000","last_peered":"2024-09-19T05:39:12.440498+0000","last_clean":"2024-09-19T05:39:12.440498+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:39:12.440498+0000","last_undegraded":"2024-09-19T05:39:12.440498+0000","last_fullsized":"2024-09-19T05:39:12.440498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439165+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:39:12.439165+0000","last_peered":"2024-09-19T05:39:12.439165+0000","last_clean":"2024-09-19T05:39:12.439165+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:39:12.439165+0000","last_undegraded":"2024-09-19T05:39:12.439165+0000","last_fullsized":"2024-09-19T05:39:12.439165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441183+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:39:12.441183+0000","last_peered":"2024-09-19T05:39:12.441183+0000","last_clean":"2024-09-19T05:39:12.441183+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:39:12.441183+0000","last_undegraded":"2024-09-19T05:39:12.441183+0000","last_fullsized":"2024-09-19T05:39:12.441183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439276+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:39:12.439276+0000","last_peered":"2024-09-19T05:39:12.439276+0000","last_clean":"2024-09-19T05:39:12.439276+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:39:12.439276+0000","last_undegraded":"2024-09-19T05:39:12.439276+0000","last_fullsized":"2024-09-19T05:39:12.439276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440261+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:39:12.440261+0000","last_peered":"2024-09-19T05:39:12.440261+0000","last_clean":"2024-09-19T05:39:12.440261+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:39:12.440261+0000","last_undegraded":"2024-09-19T05:39:12.440261+0000","last_fullsized":"2024-09-19T05:39:12.440261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439177+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:39:12.439177+0000","last_peered":"2024-09-19T05:39:12.439177+0000","last_clean":"2024-09-19T05:39:12.439177+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:39:12.439177+0000","last_undegraded":"2024-09-19T05:39:12.439177+0000","last_fullsized":"2024-09-19T05:39:12.439177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439653+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:39:12.439653+0000","last_peered":"2024-09-19T05:39:12.439653+0000","last_clean":"2024-09-19T05:39:12.439653+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:39:12.439653+0000","last_undegraded":"2024-09-19T05:39:12.439653+0000","last_fullsized":"2024-09-19T05:39:12.439653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439884+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:39:12.439884+0000","last_peered":"2024-09-19T05:39:12.439884+0000","last_clean":"2024-09-19T05:39:12.439884+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:39:12.439884+0000","last_undegraded":"2024-09-19T05:39:12.439884+0000","last_fullsized":"2024-09-19T05:39:12.439884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439672+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:39:12.439672+0000","last_peered":"2024-09-19T05:39:12.439672+0000","last_clean":"2024-09-19T05:39:12.439672+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:39:12.439672+0000","last_undegraded":"2024-09-19T05:39:12.439672+0000","last_fullsized":"2024-09-19T05:39:12.439672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.441596+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:39:12.441596+0000","last_peered":"2024-09-19T05:39:12.441596+0000","last_clean":"2024-09-19T05:39:12.441596+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:39:12.441596+0000","last_undegraded":"2024-09-19T05:39:12.441596+0000","last_fullsized":"2024-09-19T05:39:12.441596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440734+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:39:12.440734+0000","last_peered":"2024-09-19T05:39:12.440734+0000","last_clean":"2024-09-19T05:39:12.440734+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:39:12.440734+0000","last_undegraded":"2024-09-19T05:39:12.440734+0000","last_fullsized":"2024-09-19T05:39:12.440734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.438680+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:39:12.438680+0000","last_peered":"2024-09-19T05:39:12.438680+0000","last_clean":"2024-09-19T05:39:12.438680+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:39:12.438680+0000","last_undegraded":"2024-09-19T05:39:12.438680+0000","last_fullsized":"2024-09-19T05:39:12.438680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401440+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:39:12.401440+0000","last_peered":"2024-09-19T05:39:12.401440+0000","last_clean":"2024-09-19T05:39:12.401440+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:39:12.401440+0000","last_undegraded":"2024-09-19T05:39:12.401440+0000","last_fullsized":"2024-09-19T05:39:12.401440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401406+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:39:12.401406+0000","last_peered":"2024-09-19T05:39:12.401406+0000","last_clean":"2024-09-19T05:39:12.401406+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:39:12.401406+0000","last_undegraded":"2024-09-19T05:39:12.401406+0000","last_fullsized":"2024-09-19T05:39:12.401406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439574+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:39:12.439574+0000","last_peered":"2024-09-19T05:39:12.439574+0000","last_clean":"2024-09-19T05:39:12.439574+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:39:12.439574+0000","last_undegraded":"2024-09-19T05:39:12.439574+0000","last_fullsized":"2024-09-19T05:39:12.439574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397973+0000","last_change":"2024-09-19T05:39:01.903139+0000","last_active":"2024-09-19T05:39:12.397973+0000","last_peered":"2024-09-19T05:39:12.397973+0000","last_clean":"2024-09-19T05:39:12.397973+0000","last_became_active":"2024-09-19T05:39:01.902831+0000","last_became_peered":"2024-09-19T05:39:01.902831+0000","last_unstale":"2024-09-19T05:39:12.397973+0000","last_undegraded":"2024-09-19T05:39:12.397973+0000","last_fullsized":"2024-09-19T05:39:12.397973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:15.673504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440677+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:39:12.440677+0000","last_peered":"2024-09-19T05:39:12.440677+0000","last_clean":"2024-09-19T05:39:12.440677+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:39:12.440677+0000","last_undegraded":"2024-09-19T05:39:12.440677+0000","last_fullsized":"2024-09-19T05:39:12.440677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440782+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:39:12.440782+0000","last_peered":"2024-09-19T05:39:12.440782+0000","last_clean":"2024-09-19T05:39:12.440782+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:39:12.440782+0000","last_undegraded":"2024-09-19T05:39:12.440782+0000","last_fullsized":"2024-09-19T05:39:12.440782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439361+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:39:12.439361+0000","last_peered":"2024-09-19T05:39:12.439361+0000","last_clean":"2024-09-19T05:39:12.439361+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:39:12.439361+0000","last_undegraded":"2024-09-19T05:39:12.439361+0000","last_fullsized":"2024-09-19T05:39:12.439361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439287+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:39:12.439287+0000","last_peered":"2024-09-19T05:39:12.439287+0000","last_clean":"2024-09-19T05:39:12.439287+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:39:12.439287+0000","last_undegraded":"2024-09-19T05:39:12.439287+0000","last_fullsized":"2024-09-19T05:39:12.439287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439356+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:39:12.439356+0000","last_peered":"2024-09-19T05:39:12.439356+0000","last_clean":"2024-09-19T05:39:12.439356+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:39:12.439356+0000","last_undegraded":"2024-09-19T05:39:12.439356+0000","last_fullsized":"2024-09-19T05:39:12.439356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.439935+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:39:12.439935+0000","last_peered":"2024-09-19T05:39:12.439935+0000","last_clean":"2024-09-19T05:39:12.439935+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:39:12.439935+0000","last_undegraded":"2024-09-19T05:39:12.439935+0000","last_fullsized":"2024-09-19T05:39:12.439935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397137+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:39:12.397137+0000","last_peered":"2024-09-19T05:39:12.397137+0000","last_clean":"2024-09-19T05:39:12.397137+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:39:12.397137+0000","last_undegraded":"2024-09-19T05:39:12.397137+0000","last_fullsized":"2024-09-19T05:39:12.397137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.397847+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:39:12.397847+0000","last_peered":"2024-09-19T05:39:12.397847+0000","last_clean":"2024-09-19T05:39:12.397847+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:39:12.397847+0000","last_undegraded":"2024-09-19T05:39:12.397847+0000","last_fullsized":"2024-09-19T05:39:12.397847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401761+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:39:12.401761+0000","last_peered":"2024-09-19T05:39:12.401761+0000","last_clean":"2024-09-19T05:39:12.401761+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:39:12.401761+0000","last_undegraded":"2024-09-19T05:39:12.401761+0000","last_fullsized":"2024-09-19T05:39:12.401761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.401651+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:39:12.401651+0000","last_peered":"2024-09-19T05:39:12.401651+0000","last_clean":"2024-09-19T05:39:12.401651+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:39:12.401651+0000","last_undegraded":"2024-09-19T05:39:12.401651+0000","last_fullsized":"2024-09-19T05:39:12.401651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-19T05:39:12.440658+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:39:12.440658+0000","last_peered":"2024-09-19T05:39:12.440658+0000","last_clean":"2024-09-19T05:39:12.440658+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:39:12.440658+0000","last_undegraded":"2024-09-19T05:39:12.440658+0000","last_fullsized":"2024-09-19T05:39:12.440658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59893,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":182,"num_read_kb":156,"num_write":245,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":196739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547223,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":480891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1119606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939683,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":484524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":480806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1078091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":476325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1072887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":480399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":475640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":529867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":529254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":477781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:40:04.948 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:40:04.948 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T05:40:04.949 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T05:40:05.216 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:05.634 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:05 smithi038 ceph-mon[31227]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-19T05:40:05.635 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:05 smithi038 ceph-mon[31227]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.725+0000 7f0405797640 1 -- 172.21.15.38:0/2728373255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400101730 msgr2=0x7f0400101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.725+0000 7f0405797640 1 --2- 172.21.15.38:0/2728373255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400101730 0x7f0400101be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009ca0 tx=0x7f03f802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.725+0000 7f0405797640 1 -- 172.21.15.38:0/2728373255 shutdown_connections 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.725+0000 7f0405797640 1 --2- 172.21.15.38:0/2728373255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400101730 0x7f0400101be0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 --2- 172.21.15.38:0/2728373255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100d90 0x7f04001011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 --2- 172.21.15.38:0/2728373255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400100450 0x7f0400100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 -- 172.21.15.38:0/2728373255 >> 172.21.15.38:0/2728373255 conn(0x7f04000fc280 msgr2=0x7f04000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 -- 172.21.15.38:0/2728373255 shutdown_connections 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 -- 172.21.15.38:0/2728373255 wait complete. 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 Processor -- start 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.726+0000 7f0405797640 1 -- start start 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100450 0x7f040019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400101730 0x7f04001a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f040010fa90 con 0x7f0400100d90 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f040010f910 con 0x7f0400101730 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f040010fc10 con 0x7f0400100450 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:05.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48010/0 (socket says 172.21.15.38:48010) 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 -- 172.21.15.38:0/2937207823 learned_addr learned my addr 172.21.15.38:0/2937207823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03feffd640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100450 0x7f040019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03ff7fe640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400101730 0x7f04001a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 -- 172.21.15.38:0/2937207823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100450 msgr2=0x7f040019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100450 0x7f040019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 -- 172.21.15.38:0/2937207823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400101730 msgr2=0x7f04001a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400101730 0x7f04001a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 -- 172.21.15.38:0/2937207823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04001a1cb0 con 0x7f0400100d90 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03fe7fc640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03f400d770 tx=0x7f03f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4017070 con 0x7f0400100d90 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f4005500 con 0x7f0400100d90 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4004e00 con 0x7f0400100d90 2024-09-19T05:40:05.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.727+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04001a1fa0 con 0x7f0400100d90 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.728+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a2400 con 0x7f0400100d90 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.728+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040010b1a0 con 0x7f0400100d90 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.730+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f03f4004fa0 con 0x7f0400100d90 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.730+0000 7f03e7fff640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 0x7f03e80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.730+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f03f40cf6b0 con 0x7f0400100d90 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.730+0000 7f03feffd640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 0x7f03e80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.731+0000 7f03feffd640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 0x7f03e80a57c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009bd0 tx=0x7f03f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:05.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.735+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f03f409c030 con 0x7f0400100d90 2024-09-19T05:40:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:05 smithi179 ceph-mon[35781]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-19T05:40:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:05 smithi179 ceph-mon[35781]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:05 smithi047 ceph-mon[35327]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-19T05:40:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:05 smithi047 ceph-mon[35327]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:05.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.980+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0400062e90 con 0x7f0400100d90 2024-09-19T05:40:05.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.983+0000 7f03e7fff640 1 -- 172.21.15.38:0/2937207823 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f03f409b930 con 0x7f0400100d90 2024-09-19T05:40:05.984 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-19T05:40:05.985 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQA1uetmZnCHOhAAEwx+BWy/PmUdgtWIDSqLLg== 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 msgr2=0x7f03e80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 0x7f03e80a57c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009bd0 tx=0x7f03f0005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 msgr2=0x7f040019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f03f400d770 tx=0x7f03f400dc30 comp rx=0 tx=0).stop 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 shutdown_connections 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f03e80a3300 0x7f03e80a57c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0400101730 0x7f04001a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0400100d90 0x7f040019a5c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 --2- 172.21.15.38:0/2937207823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0400100450 0x7f040019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 >> 172.21.15.38:0/2937207823 conn(0x7f04000fc280 msgr2=0x7f04000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 shutdown_connections 2024-09-19T05:40:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:05.986+0000 7f0405797640 1 -- 172.21.15.38:0/2937207823 wait complete. 2024-09-19T05:40:06.631 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:40:06.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T05:40:06.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T05:40:06.670 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T05:40:06.670 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T05:40:06.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T05:40:06.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:06 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:40:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:06 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:40:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:06 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:40:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:06 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:40:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:06.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:06 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:40:06.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:06 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2937207823' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 -- 172.21.15.38:0/1035535694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0101420 msgr2=0x7f37d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 --2- 172.21.15.38:0/1035535694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0101420 0x7f37d010d950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009cd0 tx=0x7f37c802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 -- 172.21.15.38:0/1035535694 shutdown_connections 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 --2- 172.21.15.38:0/1035535694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0101420 0x7f37d010d950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 --2- 172.21.15.38:0/1035535694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0100a80 0x7f37d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.434+0000 7f37d7773640 1 --2- 172.21.15.38:0/1035535694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0106a80 0x7f37d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.435+0000 7f37d7773640 1 -- 172.21.15.38:0/1035535694 >> 172.21.15.38:0/1035535694 conn(0x7f37d00fc7a0 msgr2=0x7f37d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.435+0000 7f37d7773640 1 -- 172.21.15.38:0/1035535694 shutdown_connections 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.435+0000 7f37d7773640 1 -- 172.21.15.38:0/1035535694 wait complete. 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.435+0000 7f37d7773640 1 Processor -- start 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.435+0000 7f37d7773640 1 -- start start 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0101420 0x7f37d01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0106a80 0x7f37d019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0110e80 con 0x7f37d0100a80 2024-09-19T05:40:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0110d00 con 0x7f37d0101420 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0111000 con 0x7f37d0106a80 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52532/0 (socket says 172.21.15.38:52532) 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 -- 172.21.15.38:0/377303011 learned_addr learned my addr 172.21.15.38:0/377303011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d4d0e640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0101420 0x7f37d01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d5d10640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0106a80 0x7f37d019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 -- 172.21.15.38:0/377303011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0106a80 msgr2=0x7f37d019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0106a80 0x7f37d019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 -- 172.21.15.38:0/377303011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0101420 msgr2=0x7f37d01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0101420 0x7f37d01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 -- 172.21.15.38:0/377303011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d019d320 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d550f640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c920 tx=0x7f37cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007c70 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37cc007e10 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d019d610 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc018890 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.436+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01a4ef0 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.437+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d0100ee0 con 0x7f37d0100a80 2024-09-19T05:40:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.439+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f37cc018a50 con 0x7f37d0100a80 2024-09-19T05:40:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.440+0000 7f37c67fc640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 0x7f37bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.440+0000 7f37d4d0e640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 0x7f37bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.440+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f37cc0d0070 con 0x7f37d0100a80 2024-09-19T05:40:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.441+0000 7f37d4d0e640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 0x7f37bc0a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37d01060b0 tx=0x7f37c000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.444+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37cc014030 con 0x7f37d0100a80 2024-09-19T05:40:07.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:07 smithi038 ceph-mon[31227]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T05:40:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.669+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f37d01a51f0 con 0x7f37d0100a80 2024-09-19T05:40:07.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.674+0000 7f37c67fc640 1 -- 172.21.15.38:0/377303011 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264050 (secure 0 0 0) 0x7f37cc01f300 con 0x7f37d0100a80 2024-09-19T05:40:07.679 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.676+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 msgr2=0x7f37bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.676+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 0x7f37bc0a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37d01060b0 tx=0x7f37c000b590 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.676+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 msgr2=0x7f37d0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.676+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c920 tx=0x7f37cc00cde0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 shutdown_connections 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f37bc0a3350 0x7f37bc0a5810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37d0106a80 0x7f37d019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0101420 0x7f37d01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 --2- 172.21.15.38:0/377303011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37d0100a80 0x7f37d0198370 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 >> 172.21.15.38:0/377303011 conn(0x7f37d00fc7a0 msgr2=0x7f37d00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:07.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 shutdown_connections 2024-09-19T05:40:07.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:07.677+0000 7f37d7773640 1 -- 172.21.15.38:0/377303011 wait complete. 2024-09-19T05:40:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:07 smithi179 ceph-mon[35781]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T05:40:07.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:07 smithi047 ceph-mon[35327]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T05:40:08.410 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":4137587885},{"type":"v1","addr":"172.21.15.38:6801","nonce":4137587885}]},"active_addr":"172.21.15.38:6801/4137587885","active_change":"2024-09-19T05:32:09.496858+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3983298487}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1979840411}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2725852666}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2539325944}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2578078392}]}]} 2024-09-19T05:40:08.419 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T05:40:08.419 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T05:40:08.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:40:08.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:08 smithi038 ceph-mon[31227]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T05:40:08.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:08 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/377303011' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:40:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:08 smithi179 ceph-mon[35781]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T05:40:08.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:08 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/377303011' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:40:08.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:08 smithi047 ceph-mon[35327]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-19T05:40:08.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:08 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/377303011' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 -- 172.21.15.38:0/335493444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668102790 msgr2=0x7f26681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 --2- 172.21.15.38:0/335493444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668102790 0x7f26681067e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2658009900 tx=0x7f265802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 -- 172.21.15.38:0/335493444 shutdown_connections 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 --2- 172.21.15.38:0/335493444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668106f10 0x7f266810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 --2- 172.21.15.38:0/335493444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668102790 0x7f26681067e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 --2- 172.21.15.38:0/335493444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668101de0 0x7f26681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 -- 172.21.15.38:0/335493444 >> 172.21.15.38:0/335493444 conn(0x7f26680fd8c0 msgr2=0x7f26680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 -- 172.21.15.38:0/335493444 shutdown_connections 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.178+0000 7f26703c1640 1 -- 172.21.15.38:0/335493444 wait complete. 2024-09-19T05:40:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 Processor -- start 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 -- start start 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668101de0 0x7f26680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668102790 0x7f2668074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111f70 con 0x7f2668106f10 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111df0 con 0x7f2668102790 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f26703c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26681120f0 con 0x7f2668101de0 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f266e95e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f266e95e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52552/0 (socket says 172.21.15.38:52552) 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f266e95e640 1 -- 172.21.15.38:0/2870607310 learned_addr learned my addr 172.21.15.38:0/2870607310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f266d95c640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668102790 0x7f2668074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.179+0000 7f266e15d640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668101de0 0x7f26680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 -- 172.21.15.38:0/2870607310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668101de0 msgr2=0x7f26680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668101de0 0x7f26680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 -- 172.21.15.38:0/2870607310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668102790 msgr2=0x7f2668074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668102790 0x7f2668074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 -- 172.21.15.38:0/2870607310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26681a6990 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f266e95e640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f266000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660017070 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2660003e50 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681a6c80 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600106c0 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.180+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a7190 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.181+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26681021c0 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.183+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2660003970 con 0x7f2668106f10 2024-09-19T05:40:09.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.184+0000 7f265f7fe640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 0x7f26540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.184+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f26600cf9d0 con 0x7f2668106f10 2024-09-19T05:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.184+0000 7f266e15d640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 0x7f26540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.184+0000 7f266e15d640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 0x7f26540a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2664005fd0 tx=0x7f2664005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.187+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f266009c370 con 0x7f2668106f10 2024-09-19T05:40:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.364+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f266810a150 con 0x7f2668106f10 2024-09-19T05:40:09.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.365+0000 7f265f7fe640 1 -- 172.21.15.38:0/2870607310 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24190 (secure 0 0 0) 0x7f266009bc70 con 0x7f2668106f10 2024-09-19T05:40:09.367 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:09.367 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:39:11.677533+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 msgr2=0x7f26540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 0x7f26540a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2664005fd0 tx=0x7f2664005850 comp rx=0 tx=0).stop 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 msgr2=0x7f2668074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2660009fd0 tx=0x7f266000b260 comp rx=0 tx=0).stop 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 shutdown_connections 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f26540a3350 0x7f26540a5810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2668106f10 0x7f2668074f70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2668102790 0x7f2668074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 --2- 172.21.15.38:0/2870607310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2668101de0 0x7f26680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 >> 172.21.15.38:0/2870607310 conn(0x7f26680fd8c0 msgr2=0x7f26680fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 shutdown_connections 2024-09-19T05:40:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:09.367+0000 7f26703c1640 1 -- 172.21.15.38:0/2870607310 wait complete. 2024-09-19T05:40:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2870607310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T05:40:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-19T05:40:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-19T05:40:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2870607310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-09-19T05:40:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-19T05:40:09.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-19T05:40:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2870607310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T05:40:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T05:40:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-19T05:40:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-19T05:40:10.124 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T05:40:10.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:40:10.387 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-09-19T05:40:10.808 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-19T05:40:10.809 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-19T05:40:10.809 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:40:10.809 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:10 smithi038 ceph-mon[31227]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T05:40:10.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 -- 172.21.15.38:0/348500161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8105a90 msgr2=0x7f66a8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:10.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 --2- 172.21.15.38:0/348500161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8105a90 0x7f66a8105e70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f669802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:10.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 -- 172.21.15.38:0/348500161 shutdown_connections 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 --2- 172.21.15.38:0/348500161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 --2- 172.21.15.38:0/348500161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8101240 0x7f66a8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 --2- 172.21.15.38:0/348500161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8105a90 0x7f66a8105e70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 -- 172.21.15.38:0/348500161 >> 172.21.15.38:0/348500161 conn(0x7f66a8077210 msgr2=0x7f66a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 -- 172.21.15.38:0/348500161 shutdown_connections 2024-09-19T05:40:10.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.888+0000 7f66b02df640 1 -- 172.21.15.38:0/348500161 wait complete. 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.889+0000 7f66b02df640 1 Processor -- start 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.889+0000 7f66b02df640 1 -- start start 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8105a90 0x7f66a819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112090 con 0x7f66a8101240 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8111f10 con 0x7f66a8101bc0 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66b02df640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8112210 con 0x7f66a8105a90 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:10.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52576/0 (socket says 172.21.15.38:52576) 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ad87a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 -- 172.21.15.38:0/91620094 learned_addr learned my addr 172.21.15.38:0/91620094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae87c640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8105a90 0x7f66a819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 -- 172.21.15.38:0/91620094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8105a90 msgr2=0x7f66a819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8105a90 0x7f66a819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 -- 172.21.15.38:0/91620094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 msgr2=0x7f66a8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ae07b640 1 -- 172.21.15.38:0/91620094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a819d500 con 0x7f66a8101240 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.890+0000 7f66ad87a640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66ae07b640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6698006010 tx=0x7f6698003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:10.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669803f070 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a819d790 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a810e350 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698031c30 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.891+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698035030 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.893+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694005290 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.894+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6698042070 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.895+0000 7f66a77fe640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 0x7f66900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.895+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f66980f00c0 con 0x7f66a8101240 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.895+0000 7f66ad87a640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 0x7f66900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.895+0000 7f66ad87a640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 0x7f66900a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66a80ff730 tx=0x7f669c010860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:10.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:10.899+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6698031de0 con 0x7f66a8101240 2024-09-19T05:40:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T05:40:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T05:40:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:40:10.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:10 smithi179 ceph-mon[35781]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-19T05:40:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-09-19T05:40:10.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-19T05:40:10.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-19T05:40:10.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:40:10.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:10 smithi047 ceph-mon[35327]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T05:40:11.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.076+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6694005850 con 0x7f66a8101240 2024-09-19T05:40:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.078+0000 7f66a77fe640 1 -- 172.21.15.38:0/91620094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24660 (secure 0 0 0) 0x7f66980bc2d0 con 0x7f66a8101240 2024-09-19T05:40:11.080 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:11.080 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":89,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:40:10.573456+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 msgr2=0x7f66900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 0x7f66900a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66a80ff730 tx=0x7f669c010860 comp rx=0 tx=0).stop 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 msgr2=0x7f66a8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6698006010 tx=0x7f6698003c10 comp rx=0 tx=0).stop 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 shutdown_connections 2024-09-19T05:40:11.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f66900a3350 0x7f66900a5810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66a8105a90 0x7f66a819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a8101bc0 0x7f66a8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 --2- 172.21.15.38:0/91620094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8101240 0x7f66a8198550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 >> 172.21.15.38:0/91620094 conn(0x7f66a8077210 msgr2=0x7f66a8103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.080+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 shutdown_connections 2024-09-19T05:40:11.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:11.081+0000 7f66b02df640 1 -- 172.21.15.38:0/91620094 wait complete. 2024-09-19T05:40:11.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:11 smithi038 ceph-mon[31227]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:40:11.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:11 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/91620094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:11.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T05:40:11.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T05:40:11.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T05:40:11.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T05:40:11.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T05:40:11.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T05:40:11.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T05:40:11.807 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T05:40:11.807 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T05:40:11.807 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T05:40:11.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T05:40:11.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T05:40:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:11 smithi179 ceph-mon[35781]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:40:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:11 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/91620094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:11.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:11 smithi047 ceph-mon[35327]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:40:11.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:11 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/91620094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:12.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:12 smithi038 ceph-mon[31227]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-19T05:40:12.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:12 smithi179 ceph-mon[35781]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-19T05:40:12.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:12 smithi047 ceph-mon[35327]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-19T05:40:13.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 -- 172.21.15.38:0/1482952414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0102dd0 msgr2=0x7ff1a01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1482952414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0102dd0 0x7ff1a01031b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff190009930 tx=0x7ff19002a420 comp rx=0 tx=0).stop 2024-09-19T05:40:13.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 -- 172.21.15.38:0/1482952414 shutdown_connections 2024-09-19T05:40:13.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1482952414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0108300 0x7ff1a010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1482952414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1a0103780 0x7ff1a0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1482952414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0102dd0 0x7ff1a01031b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 -- 172.21.15.38:0/1482952414 >> 172.21.15.38:0/1482952414 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 -- 172.21.15.38:0/1482952414 shutdown_connections 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.084+0000 7ff1a681f640 1 -- 172.21.15.38:0/1482952414 wait complete. 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 Processor -- start 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 -- start start 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0102dd0 0x7ff1a0103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1a0103780 0x7ff1a0104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 0x7ff1a01046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0112090 con 0x7ff1a0108300 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111f10 con 0x7ff1a0103780 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.085+0000 7ff1a681f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0112210 con 0x7ff1a0102dd0 2024-09-19T05:40:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 0x7ff1a01046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:13.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0102dd0 0x7ff1a0103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:13.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff197fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1a0103780 0x7ff1a0104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:13.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0102dd0 0x7ff1a0103c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:54622/0 (socket says 172.21.15.38:54622) 2024-09-19T05:40:13.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 0x7ff1a01046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52588/0 (socket says 172.21.15.38:52588) 2024-09-19T05:40:13.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19ffff640 1 -- 172.21.15.38:0/1525850660 learned_addr learned my addr 172.21.15.38:0/1525850660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 -- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0102dd0 msgr2=0x7ff1a0103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1a0102dd0 0x7ff1a0103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 -- 172.21.15.38:0/1525850660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1a0103780 msgr2=0x7ff1a0104190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1a0103780 0x7ff1a0104190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a01aae00 con 0x7ff1a0108300 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a4dbc640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 0x7ff1a01046d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff19800cae0 tx=0x7ff19800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19800ae30 con 0x7ff1a0108300 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a01ab0f0 con 0x7ff1a0108300 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19800f930 con 0x7ff1a0108300 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198018af0 con 0x7ff1a0108300 2024-09-19T05:40:13.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.086+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01ab630 con 0x7ff1a0108300 2024-09-19T05:40:13.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.087+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1a0062980 con 0x7ff1a0108300 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.088+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff19800fae0 con 0x7ff1a0108300 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19dffb640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff18c0a32d0 0x7ff18c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19ffff640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff18c0a32d0 0x7ff18c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff1980d0390 con 0x7ff1a0108300 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19dffb640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] conn(0x7ff18c0ae4a0 0x7ff18c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff198016e80 con 0x7ff18c0ae4a0 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.089+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff1980d7050 con 0x7ff1a0108300 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.090+0000 7ff19ffff640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff18c0a32d0 0x7ff18c0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1900096c0 tx=0x7ff190005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.090+0000 7ff197fff640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] conn(0x7ff18c0ae4a0 0x7ff18c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.090+0000 7ff197fff640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] conn(0x7ff18c0ae4a0 0x7ff18c0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.091+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== osd.2 v2:172.21.15.179:6800/4020882332 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff198016e80 con 0x7ff18c0ae4a0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.105+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 --> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1a0062e30 con 0x7ff18c0ae4a0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.105+0000 7ff19dffb640 1 -- 172.21.15.38:0/1525850660 <== osd.2 v2:172.21.15.179:6800/4020882332 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1a0062e30 con 0x7ff18c0ae4a0 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] conn(0x7ff18c0ae4a0 msgr2=0x7ff18c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] conn(0x7ff18c0ae4a0 0x7ff18c0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff18c0a32d0 msgr2=0x7ff18c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff18c0a32d0 0x7ff18c0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1900096c0 tx=0x7ff190005df0 comp rx=0 tx=0).stop 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 msgr2=0x7ff1a01046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 --2- 172.21.15.38:0/1525850660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a0108300 0x7ff1a01046d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff19800cae0 tx=0x7ff19800a250 comp rx=0 tx=0).stop 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a4dbc640 1 -- 172.21.15.38:0/1525850660 reap_dead start 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 shutdown_connections 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 >> 172.21.15.38:0/1525850660 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 shutdown_connections 2024-09-19T05:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:13.106+0000 7ff1a681f640 1 -- 172.21.15.38:0/1525850660 wait complete. 2024-09-19T05:40:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:13 smithi038 ceph-mon[31227]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T05:40:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:13 smithi047 ceph-mon[35327]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T05:40:14.002 INFO:teuthology.orchestra.run.smithi038.stdout:77309411401 2024-09-19T05:40:14.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T05:40:14.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:13 smithi179 ceph-mon[35781]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T05:40:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 -- 172.21.15.38:0/1955023481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c102d30 msgr2=0x7f550c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 --2- 172.21.15.38:0/1955023481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c102d30 0x7f550c103110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009930 tx=0x7f54fc02a420 comp rx=0 tx=0).stop 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 -- 172.21.15.38:0/1955023481 shutdown_connections 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 --2- 172.21.15.38:0/1955023481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550c1082a0 0x7f550c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 --2- 172.21.15.38:0/1955023481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1036e0 0x7f550c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 --2- 172.21.15.38:0/1955023481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c102d30 0x7f550c103110 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 -- 172.21.15.38:0/1955023481 >> 172.21.15.38:0/1955023481 conn(0x7f550c0fc7d0 msgr2=0x7f550c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 -- 172.21.15.38:0/1955023481 shutdown_connections 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.173+0000 7f55108d4640 1 -- 172.21.15.38:0/1955023481 wait complete. 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 Processor -- start 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 -- start start 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550c102d30 0x7f550c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 0x7f550c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1082a0 0x7f550c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1120a0 con 0x7f550c1036e0 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f550c111f20 con 0x7f550c102d30 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.174+0000 7f55108d4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f550c112220 con 0x7f550c1082a0 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 0x7f550c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 0x7f550c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52608/0 (socket says 172.21.15.38:52608) 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f550a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550c102d30 0x7f550c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 -- 172.21.15.38:0/1559563815 learned_addr learned my addr 172.21.15.38:0/1559563815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f550ad9d640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1082a0 0x7f550c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 -- 172.21.15.38:0/1559563815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1082a0 msgr2=0x7f550c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1082a0 0x7f550c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 -- 172.21.15.38:0/1559563815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550c102d30 msgr2=0x7f550c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f550c102d30 0x7f550c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c19d5d0 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f550ad9d640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f550c1082a0 0x7f550c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f5509d9b640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 0x7f550c198b60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f550000d750 tx=0x7f550000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500017070 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5500005500 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c19d8c0 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.176+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c1a51a0 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.175+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500004e00 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.177+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f550c062980 con 0x7f550c1036e0 2024-09-19T05:40:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.178+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5500004fa0 con 0x7f550c1036e0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.178+0000 7f54fb7fe640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54f40a32d0 0x7f54f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.178+0000 7f550a59c640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54f40a32d0 0x7f54f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.178+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55000cf7c0 con 0x7f550c1036e0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f54fb7fe640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] conn(0x7f54f40ae4a0 0x7f54f40b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54f40b1010 con 0x7f54f40ae4a0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f55000d6050 con 0x7f550c1036e0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f550ad9d640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] conn(0x7f54f40ae4a0 0x7f54f40b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f550a59c640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54f40a32d0 0x7f54f40a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009900 tx=0x7f54fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.179+0000 7f550ad9d640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] conn(0x7f54f40ae4a0 0x7f54f40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.180+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== osd.3 v2:172.21.15.38:6810/1962488545 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54f40b1010 con 0x7f54f40ae4a0 2024-09-19T05:40:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.202+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 --> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f550c062d80 con 0x7f54f40ae4a0 2024-09-19T05:40:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.202+0000 7f54fb7fe640 1 -- 172.21.15.38:0/1559563815 <== osd.3 v2:172.21.15.38:6810/1962488545 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f550c062d80 con 0x7f54f40ae4a0 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] conn(0x7f54f40ae4a0 msgr2=0x7f54f40b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] conn(0x7f54f40ae4a0 0x7f54f40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54f40a32d0 msgr2=0x7f54f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54f40a32d0 0x7f54f40a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009900 tx=0x7f54fc0058d0 comp rx=0 tx=0).stop 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 msgr2=0x7f550c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 --2- 172.21.15.38:0/1559563815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c1036e0 0x7f550c198b60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f550000d750 tx=0x7f550000dc10 comp rx=0 tx=0).stop 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f550ad9d640 1 -- 172.21.15.38:0/1559563815 reap_dead start 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 shutdown_connections 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 >> 172.21.15.38:0/1559563815 conn(0x7f550c0fc7d0 msgr2=0x7f550c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 shutdown_connections 2024-09-19T05:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:14.203+0000 7f55108d4640 1 -- 172.21.15.38:0/1559563815 wait complete. 2024-09-19T05:40:14.438 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.438 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.442 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.448 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:14.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:14 smithi038 ceph-mon[31227]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-19T05:40:14.953 INFO:teuthology.orchestra.run.smithi038.stdout:107374182468 2024-09-19T05:40:14.954 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T05:40:14.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:14 smithi047 ceph-mon[35327]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-19T05:40:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:14 smithi179 ceph-mon[35781]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-19T05:40:15.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:15 smithi038 ceph-mon[31227]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T05:40:15.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:15 smithi038 ceph-mon[31227]: Cluster is now healthy 2024-09-19T05:40:15.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 -- 172.21.15.38:0/2228417646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8101420 msgr2=0x7f6ab810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 --2- 172.21.15.38:0/2228417646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8101420 0x7f6ab810d950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009cd0 tx=0x7f6ab002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 -- 172.21.15.38:0/2228417646 shutdown_connections 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 --2- 172.21.15.38:0/2228417646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8101420 0x7f6ab810d950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 --2- 172.21.15.38:0/2228417646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab8100a80 0x7f6ab8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 --2- 172.21.15.38:0/2228417646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8106a80 0x7f6ab8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 -- 172.21.15.38:0/2228417646 >> 172.21.15.38:0/2228417646 conn(0x7f6ab80fc7a0 msgr2=0x7f6ab80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 -- 172.21.15.38:0/2228417646 shutdown_connections 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.881+0000 7f6abe872640 1 -- 172.21.15.38:0/2228417646 wait complete. 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 Processor -- start 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 -- start start 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8100a80 0x7f6ab819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab8101420 0x7f6ab819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:15.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 0x7f6ab819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8111fa0 con 0x7f6ab8106a80 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8111e20 con 0x7f6ab8101420 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abe872640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8112120 con 0x7f6ab8100a80 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abce0f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 0x7f6ab819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abce0f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 0x7f6ab819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52618/0 (socket says 172.21.15.38:52618) 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.882+0000 7f6abce0f640 1 -- 172.21.15.38:0/2842546012 learned_addr learned my addr 172.21.15.38:0/2842546012 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6ab7fff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8100a80 0x7f6ab819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6aaffff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab8101420 0x7f6ab819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 -- 172.21.15.38:0/2842546012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8100a80 msgr2=0x7f6ab819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8100a80 0x7f6ab819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 -- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab8101420 msgr2=0x7f6ab819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ab8101420 0x7f6ab819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab819f630 con 0x7f6ab8106a80 2024-09-19T05:40:15.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6ab7fff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab8100a80 0x7f6ab819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abce0f640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 0x7f6ab819ef50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006010 tx=0x7f6ab0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004450 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab819f8c0 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab0003e10 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00466e0 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.883+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81a7200 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.884+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ab80fe7f0 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.885+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6ab00305a0 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab5ffb640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6aa40a32d0 0x7f6aa40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6ab002e030 con 0x7f6ab8106a80 2024-09-19T05:40:15.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab7fff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6aa40a32d0 0x7f6aa40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab5ffb640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] conn(0x7f6aa40ae4a0 0x7f6aa40b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6aa40b1010 con 0x7f6aa40ae4a0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.886+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f6ab00f8050 con 0x7f6ab8106a80 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.887+0000 7f6ab7fff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6aa40a32d0 0x7f6aa40a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009930 tx=0x7f6aa8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.887+0000 7f6aaffff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] conn(0x7f6aa40ae4a0 0x7f6aa40b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.887+0000 7f6aaffff640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] conn(0x7f6aa40ae4a0 0x7f6aa40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.888+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== osd.1 v2:172.21.15.47:6800/716320607 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6aa40b1010 con 0x7f6aa40ae4a0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.902+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 --> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ab81021c0 con 0x7f6aa40ae4a0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6ab5ffb640 1 -- 172.21.15.38:0/2842546012 <== osd.1 v2:172.21.15.47:6800/716320607 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6ab81021c0 con 0x7f6aa40ae4a0 2024-09-19T05:40:15.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] conn(0x7f6aa40ae4a0 msgr2=0x7f6aa40b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] conn(0x7f6aa40ae4a0 0x7f6aa40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6aa40a32d0 msgr2=0x7f6aa40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6aa40a32d0 0x7f6aa40a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009930 tx=0x7f6aa8008040 comp rx=0 tx=0).stop 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 msgr2=0x7f6ab819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 --2- 172.21.15.38:0/2842546012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8106a80 0x7f6ab819ef50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006010 tx=0x7f6ab0003c20 comp rx=0 tx=0).stop 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abce0f640 1 -- 172.21.15.38:0/2842546012 reap_dead start 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 shutdown_connections 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 >> 172.21.15.38:0/2842546012 conn(0x7f6ab80fc7a0 msgr2=0x7f6ab80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 shutdown_connections 2024-09-19T05:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:15.903+0000 7f6abe872640 1 -- 172.21.15.38:0/2842546012 wait complete. 2024-09-19T05:40:16.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:15 smithi179 ceph-mon[35781]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T05:40:16.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:15 smithi179 ceph-mon[35781]: Cluster is now healthy 2024-09-19T05:40:16.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:15 smithi047 ceph-mon[35327]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T05:40:16.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:15 smithi047 ceph-mon[35327]: Cluster is now healthy 2024-09-19T05:40:16.586 INFO:teuthology.orchestra.run.smithi038.stdout:55834574927 2024-09-19T05:40:16.587 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T05:40:16.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:16 smithi038 ceph-mon[31227]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T05:40:17.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:16 smithi179 ceph-mon[35781]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T05:40:17.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:16 smithi047 ceph-mon[35327]: pgmap v322: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-19T05:40:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 -- 172.21.15.38:0/1229148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 msgr2=0x7f806010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 --2- 172.21.15.38:0/1229148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806010ea90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8058009ca0 tx=0x7f805802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 -- 172.21.15.38:0/1229148361 shutdown_connections 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 --2- 172.21.15.38:0/1229148361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806010ea90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 --2- 172.21.15.38:0/1229148361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8060102790 0x7f80601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 --2- 172.21.15.38:0/1229148361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060101de0 0x7f80601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 -- 172.21.15.38:0/1229148361 >> 172.21.15.38:0/1229148361 conn(0x7f80600fd8c0 msgr2=0x7f80600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 -- 172.21.15.38:0/1229148361 shutdown_connections 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.436+0000 7f80679d3640 1 -- 172.21.15.38:0/1229148361 wait complete. 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 Processor -- start 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 -- start start 2024-09-19T05:40:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8060101de0 0x7f8060198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060102790 0x7f8060198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8060111f70 con 0x7f8060106f10 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8060111df0 con 0x7f8060102790 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f80679d3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80601120f0 con 0x7f8060101de0 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f8065f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f8065f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50616/0 (socket says 172.21.15.38:50616) 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f8065f70640 1 -- 172.21.15.38:0/2254137906 learned_addr learned my addr 172.21.15.38:0/2254137906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f8064f6e640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060102790 0x7f8060198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.437+0000 7f806576f640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8060101de0 0x7f8060198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 -- 172.21.15.38:0/2254137906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8060101de0 msgr2=0x7f8060198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8060101de0 0x7f8060198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 -- 172.21.15.38:0/2254137906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060102790 msgr2=0x7f8060198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8060102790 0x7f8060198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806019d2e0 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f8065f70640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806019cc00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8058031a20 tx=0x7f8058031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058003b70 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8058003d10 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806019d570 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805802fc80 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.438+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80601a4eb0 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.439+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80600feea0 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.440+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f80580046a0 con 0x7f8060106f10 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f80567fc640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f804c0a3350 0x7f804c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f806576f640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f804c0a3350 0x7f804c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f80580f1160 con 0x7f8060106f10 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f80567fc640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] conn(0x7f804c0ae520 0x7f804c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f804c0b1090 con 0x7f804c0ae520 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f80580f1510 con 0x7f8060106f10 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f8064f6e640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] conn(0x7f804c0ae520 0x7f804c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.441+0000 7f806576f640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f804c0a3350 0x7f804c0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f805c005fd0 tx=0x7f805c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.442+0000 7f8064f6e640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] conn(0x7f804c0ae520 0x7f804c0b09c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.442+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== osd.6 v2:172.21.15.38:6818/2208960184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f804c0b1090 con 0x7f804c0ae520 2024-09-19T05:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.456+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 --> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80601032c0 con 0x7f804c0ae520 2024-09-19T05:40:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.456+0000 7f80567fc640 1 -- 172.21.15.38:0/2254137906 <== osd.6 v2:172.21.15.38:6818/2208960184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f80601032c0 con 0x7f804c0ae520 2024-09-19T05:40:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] conn(0x7f804c0ae520 msgr2=0x7f804c0b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] conn(0x7f804c0ae520 0x7f804c0b09c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f804c0a3350 msgr2=0x7f804c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f804c0a3350 0x7f804c0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f805c005fd0 tx=0x7f805c005850 comp rx=0 tx=0).stop 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 msgr2=0x7f806019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 --2- 172.21.15.38:0/2254137906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8060106f10 0x7f806019cc00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8058031a20 tx=0x7f8058031a50 comp rx=0 tx=0).stop 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f8065f70640 1 -- 172.21.15.38:0/2254137906 reap_dead start 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 shutdown_connections 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 >> 172.21.15.38:0/2254137906 conn(0x7f80600fd8c0 msgr2=0x7f80600ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 shutdown_connections 2024-09-19T05:40:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:17.457+0000 7f80679d3640 1 -- 172.21.15.38:0/2254137906 wait complete. 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.194+0000 7f8bc50cf640 1 -- 172.21.15.38:0/2782780049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0101de0 msgr2=0x7f8bc01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.194+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/2782780049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0101de0 0x7f8bc01021c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb002a410 comp rx=0 tx=0).stop 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.194+0000 7f8bc50cf640 1 -- 172.21.15.38:0/2782780049 shutdown_connections 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.194+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/2782780049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bc0106f10 0x7f8bc010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/2782780049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bc0102790 0x7f8bc01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/2782780049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0101de0 0x7f8bc01021c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 -- 172.21.15.38:0/2782780049 >> 172.21.15.38:0/2782780049 conn(0x7f8bc00fd8c0 msgr2=0x7f8bc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 -- 172.21.15.38:0/2782780049 shutdown_connections 2024-09-19T05:40:18.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 -- 172.21.15.38:0/2782780049 wait complete. 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 Processor -- start 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.195+0000 7f8bc50cf640 1 -- start start 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bc0101de0 0x7f8bc0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 0x7f8bc01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bc0106f10 0x7f8bc019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0111f30 con 0x7f8bc0102790 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0111db0 con 0x7f8bc0101de0 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bc50cf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc01120b0 con 0x7f8bc0106f10 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 0x7f8bc01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 0x7f8bc01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50660/0 (socket says 172.21.15.38:50660) 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 -- 172.21.15.38:0/404748099 learned_addr learned my addr 172.21.15.38:0/404748099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbf59e640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bc0106f10 0x7f8bc019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbed9d640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bc0101de0 0x7f8bc0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 -- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bc0106f10 msgr2=0x7f8bc019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8bc0106f10 0x7f8bc019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 -- 172.21.15.38:0/404748099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bc0101de0 msgr2=0x7f8bc0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bc0101de0 0x7f8bc0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc019d440 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.196+0000 7f8bbe59c640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 0x7f8bc01989d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4017030 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc019d730 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4004020 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40047a0 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01a5010 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.197+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bc0062980 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.199+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8bb40041d0 con 0x7f8bc0102790 2024-09-19T05:40:18.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8ba7fff640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ba80a32d0 0x7f8ba80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8bbed9d640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ba80a32d0 0x7f8ba80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8bb40cfe20 con 0x7f8bc0102790 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8ba7fff640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] conn(0x7f8ba80ae4a0 0x7f8ba80b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ba80b1010 con 0x7f8ba80ae4a0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8bb40d0270 con 0x7f8bc0102790 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8bbed9d640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ba80a32d0 0x7f8ba80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.200+0000 7f8bbf59e640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] conn(0x7f8ba80ae4a0 0x7f8ba80b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.201+0000 7f8bbf59e640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] conn(0x7f8ba80ae4a0 0x7f8ba80b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.201+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== osd.11 v2:172.21.15.179:6824/1092638646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ba80b1010 con 0x7f8ba80ae4a0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.215+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 --> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bc0062d80 con 0x7f8ba80ae4a0 2024-09-19T05:40:18.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8ba7fff640 1 -- 172.21.15.38:0/404748099 <== osd.11 v2:172.21.15.179:6824/1092638646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8bc0062d80 con 0x7f8ba80ae4a0 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] conn(0x7f8ba80ae4a0 msgr2=0x7f8ba80b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] conn(0x7f8ba80ae4a0 0x7f8ba80b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ba80a32d0 msgr2=0x7f8ba80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ba80a32d0 0x7f8ba80a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009230 tx=0x7f8bb00058d0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 msgr2=0x7f8bc01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 --2- 172.21.15.38:0/404748099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bc0102790 0x7f8bc01989d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).stop 2024-09-19T05:40:18.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bbf59e640 1 -- 172.21.15.38:0/404748099 reap_dead start 2024-09-19T05:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 shutdown_connections 2024-09-19T05:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 >> 172.21.15.38:0/404748099 conn(0x7f8bc00fd8c0 msgr2=0x7f8bc00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 shutdown_connections 2024-09-19T05:40:18.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.216+0000 7f8bc50cf640 1 -- 172.21.15.38:0/404748099 wait complete. 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 -- 172.21.15.38:0/2793850884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614102790 msgr2=0x7fd6141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 --2- 172.21.15.38:0/2793850884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614102790 0x7fd6141067e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc02a410 comp rx=0 tx=0).stop 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 -- 172.21.15.38:0/2793850884 shutdown_connections 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 --2- 172.21.15.38:0/2793850884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614106f10 0x7fd61410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 --2- 172.21.15.38:0/2793850884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614102790 0x7fd6141067e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 --2- 172.21.15.38:0/2793850884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd614101de0 0x7fd6141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 -- 172.21.15.38:0/2793850884 >> 172.21.15.38:0/2793850884 conn(0x7fd6140fd8c0 msgr2=0x7fd6140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 -- 172.21.15.38:0/2793850884 shutdown_connections 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.074+0000 7fd61a094640 1 -- 172.21.15.38:0/2793850884 wait complete. 2024-09-19T05:40:18.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 Processor -- start 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 -- start start 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd614101de0 0x7fd6141984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614102790 0x7fd614198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 0x7fd61419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd614111fb0 con 0x7fd614106f10 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd614111e30 con 0x7fd614102790 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd61a094640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd614112130 con 0x7fd614101de0 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd613fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 0x7fd61419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd613fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 0x7fd61419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50634/0 (socket says 172.21.15.38:50634) 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd613fff640 1 -- 172.21.15.38:0/2272123303 learned_addr learned my addr 172.21.15.38:0/2272123303 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd6137fe640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd614101de0 0x7fd6141984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.075+0000 7fd60bfff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614102790 0x7fd614198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 -- 172.21.15.38:0/2272123303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd614101de0 msgr2=0x7fd6141984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd614101de0 0x7fd6141984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 -- 172.21.15.38:0/2272123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614102790 msgr2=0x7fd614198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614102790 0x7fd614198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61419d480 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd60bfff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd614102790 0x7fd614198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd613fff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 0x7fd61419cda0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b970 tx=0x7fd60c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c019070 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c00cb40 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c007650 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61419d770 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.076+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a5050 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.077+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd614062980 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.078+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd60c01c070 con 0x7fd614106f10 2024-09-19T05:40:18.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6117fa640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd6000a3350 0x7fd6000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6137fe640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd6000a3350 0x7fd6000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd60c0cfa60 con 0x7fd614106f10 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6117fa640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] conn(0x7fd6000ae520 0x7fd6000b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd60c0047d0 con 0x7fd6000ae520 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd60c0cfeb0 con 0x7fd614106f10 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd60bfff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] conn(0x7fd6000ae520 0x7fd6000b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.079+0000 7fd6137fe640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd6000a3350 0x7fd6000a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd604005fd0 tx=0x7fd60400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.080+0000 7fd60bfff640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] conn(0x7fd6000ae520 0x7fd6000b09c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.080+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== osd.9 v2:172.21.15.38:6826/3187320508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd60c0047d0 con 0x7fd6000ae520 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 --> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd614062e00 con 0x7fd6000ae520 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd6117fa640 1 -- 172.21.15.38:0/2272123303 <== osd.9 v2:172.21.15.38:6826/3187320508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd614062e00 con 0x7fd6000ae520 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] conn(0x7fd6000ae520 msgr2=0x7fd6000b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] conn(0x7fd6000ae520 0x7fd6000b09c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd6000a3350 msgr2=0x7fd6000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd6000a3350 0x7fd6000a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd604005fd0 tx=0x7fd60400b390 comp rx=0 tx=0).stop 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 msgr2=0x7fd61419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd61a094640 1 --2- 172.21.15.38:0/2272123303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd614106f10 0x7fd61419cda0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b970 tx=0x7fd60c00be30 comp rx=0 tx=0).stop 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.094+0000 7fd613fff640 1 -- 172.21.15.38:0/2272123303 reap_dead start 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.095+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 shutdown_connections 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.095+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 >> 172.21.15.38:0/2272123303 conn(0x7fd6140fd8c0 msgr2=0x7fd6140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.095+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 shutdown_connections 2024-09-19T05:40:18.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.095+0000 7fd61a094640 1 -- 172.21.15.38:0/2272123303 wait complete. 2024-09-19T05:40:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/388254862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4101de0 msgr2=0x7f8cf41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/388254862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4101de0 0x7f8cf41021c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009ca0 tx=0x7f8ce402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/388254862 shutdown_connections 2024-09-19T05:40:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/388254862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf4106f10 0x7f8cf410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/388254862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf4102790 0x7f8cf41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/388254862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4101de0 0x7f8cf41021c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/388254862 >> 172.21.15.38:0/388254862 conn(0x7f8cf40fd8e0 msgr2=0x7f8cf40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/388254862 shutdown_connections 2024-09-19T05:40:18.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.892+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/388254862 wait complete. 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 Processor -- start 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 -- start start 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf4101de0 0x7f8cf419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf4102790 0x7f8cf419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 0x7f8cf419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4111fc0 con 0x7f8cf4106f10 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4111e40 con 0x7f8cf4101de0 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf9ba3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4112140 con 0x7f8cf4102790 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 0x7f8cf419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 0x7f8cf419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50680/0 (socket says 172.21.15.38:50680) 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf3fff640 1 -- 172.21.15.38:0/2876968478 learned_addr learned my addr 172.21.15.38:0/2876968478 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:18.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf37fe640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf4101de0 0x7f8cf419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.893+0000 7f8cf2ffd640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf4102790 0x7f8cf419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 -- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf4102790 msgr2=0x7f8cf419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf4102790 0x7f8cf419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 -- 172.21.15.38:0/2876968478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf4101de0 msgr2=0x7f8cf419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf4101de0 0x7f8cf419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf419f670 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf3fff640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 0x7f8cf419ef90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00c970 tx=0x7f8cec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec007bb0 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cec007d50 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf419f960 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00f450 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.894+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf41a7240 con 0x7f8cf4106f10 2024-09-19T05:40:18.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.895+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cf410e0d0 con 0x7f8cf4106f10 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.896+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8cec004f40 con 0x7f8cf4106f10 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf0ff9640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8cdc0a32d0 0x7f8cdc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf37fe640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8cdc0a32d0 0x7f8cdc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8cec0cfdd0 con 0x7f8cf4106f10 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf0ff9640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] conn(0x7f8cdc0ae4a0 0x7f8cdc0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cec012ec0 con 0x7f8cdc0ae4a0 2024-09-19T05:40:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.897+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8cec0d0220 con 0x7f8cf4106f10 2024-09-19T05:40:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.898+0000 7f8cf37fe640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8cdc0a32d0 0x7f8cdc0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4006010 tx=0x7f8ce4009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.898+0000 7f8cf2ffd640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] conn(0x7f8cdc0ae4a0 0x7f8cdc0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.898+0000 7f8cf2ffd640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] conn(0x7f8cdc0ae4a0 0x7f8cdc0b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:18.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.899+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== osd.5 v2:172.21.15.179:6808/322546150 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cec012ec0 con 0x7f8cdc0ae4a0 2024-09-19T05:40:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.917+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 --> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cf41032c0 con 0x7f8cdc0ae4a0 2024-09-19T05:40:18.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.918+0000 7f8cf0ff9640 1 -- 172.21.15.38:0/2876968478 <== osd.5 v2:172.21.15.179:6808/322546150 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8cf41032c0 con 0x7f8cdc0ae4a0 2024-09-19T05:40:18.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] conn(0x7f8cdc0ae4a0 msgr2=0x7f8cdc0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] conn(0x7f8cdc0ae4a0 0x7f8cdc0b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8cdc0a32d0 msgr2=0x7f8cdc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8cdc0a32d0 0x7f8cdc0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4006010 tx=0x7f8ce4009820 comp rx=0 tx=0).stop 2024-09-19T05:40:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 msgr2=0x7f8cf419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 --2- 172.21.15.38:0/2876968478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf4106f10 0x7f8cf419ef90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00c970 tx=0x7f8cec00ce30 comp rx=0 tx=0).stop 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf3fff640 1 -- 172.21.15.38:0/2876968478 reap_dead start 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 shutdown_connections 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.919+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 >> 172.21.15.38:0/2876968478 conn(0x7f8cf40fd8e0 msgr2=0x7f8cf40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.920+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 shutdown_connections 2024-09-19T05:40:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:18.920+0000 7f8cf9ba3640 1 -- 172.21.15.38:0/2876968478 wait complete. 2024-09-19T05:40:19.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:18 smithi038 ceph-mon[31227]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 638 B/s wr, 12 op/s 2024-09-19T05:40:19.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:18 smithi179 ceph-mon[35781]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 638 B/s wr, 12 op/s 2024-09-19T05:40:19.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:18 smithi047 ceph-mon[35327]: pgmap v323: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 638 B/s wr, 12 op/s 2024-09-19T05:40:19.784 INFO:teuthology.orchestra.run.smithi038.stdout:188978561078 2024-09-19T05:40:19.784 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/1329960485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 msgr2=0x7fcbd0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/1329960485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0107b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009930 tx=0x7fcbc402a420 comp rx=0 tx=0).stop 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/1329960485 shutdown_connections 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/1329960485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0108290 0x7fcbd010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/1329960485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0107b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/1329960485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd0102d90 0x7fcbd0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/1329960485 >> 172.21.15.38:0/1329960485 conn(0x7fcbd00fc7d0 msgr2=0x7fcbd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:20.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/1329960485 shutdown_connections 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/1329960485 wait complete. 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.743+0000 7fcbd5dfe640 1 Processor -- start 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 -- start start 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0102d90 0x7fcbd01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd0108290 0x7fcbd0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0112090 con 0x7fcbd0103740 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0111f10 con 0x7fcbd0102d90 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbd5dfe640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0112210 con 0x7fcbd0108290 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50698/0 (socket says 172.21.15.38:50698) 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 -- 172.21.15.38:0/4288655947 learned_addr learned my addr 172.21.15.38:0/4288655947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbcf7fe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0102d90 0x7fcbd01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbcffff640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd0108290 0x7fcbd0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 -- 172.21.15.38:0/4288655947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd0108290 msgr2=0x7fcbd0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd0108290 0x7fcbd0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 -- 172.21.15.38:0/4288655947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0102d90 msgr2=0x7fcbd01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0102d90 0x7fcbd01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbceffd640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01a6ad0 con 0x7fcbd0103740 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.744+0000 7fcbcf7fe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd0102d90 0x7fcbd01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbceffd640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0100f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4005940 tx=0x7fcbc40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc403f030 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc4003ce0 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01a6d60 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4035070 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.745+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01a7230 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.746+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcbd00628c0 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.747+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcbc4042030 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.748+0000 7fcbccff9640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbb80a3350 0x7fcbb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.748+0000 7fcbcf7fe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbb80a3350 0x7fcbb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.748+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcbc40f0a70 con 0x7fcbd0103740 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbccff9640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] conn(0x7fcbb80ae520 0x7fcbb80b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbc4008e50 con 0x7fcbb80ae520 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fcbc40f0e20 con 0x7fcbd0103740 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbcffff640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] conn(0x7fcbb80ae520 0x7fcbb80b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbcf7fe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbb80a3350 0x7fcbb80a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0005fd0 tx=0x7fcbc0009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.749+0000 7fcbcffff640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] conn(0x7fcbb80ae520 0x7fcbb80b09c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.750+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== osd.0 v2:172.21.15.38:6802/3037922100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbc4008e50 con 0x7fcbb80ae520 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.765+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 --> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcbd0062cc0 con 0x7fcbb80ae520 2024-09-19T05:40:20.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.766+0000 7fcbccff9640 1 -- 172.21.15.38:0/4288655947 <== osd.0 v2:172.21.15.38:6802/3037922100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcbd0062cc0 con 0x7fcbb80ae520 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.766+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] conn(0x7fcbb80ae520 msgr2=0x7fcbb80b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.766+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] conn(0x7fcbb80ae520 0x7fcbb80b09c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.766+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbb80a3350 msgr2=0x7fcbb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.766+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbb80a3350 0x7fcbb80a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0005fd0 tx=0x7fcbc0009520 comp rx=0 tx=0).stop 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 msgr2=0x7fcbd0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 --2- 172.21.15.38:0/4288655947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbd0103740 0x7fcbd0100f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4005940 tx=0x7fcbc40042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbcffff640 1 -- 172.21.15.38:0/4288655947 reap_dead start 2024-09-19T05:40:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 shutdown_connections 2024-09-19T05:40:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 >> 172.21.15.38:0/4288655947 conn(0x7fcbd00fc7d0 msgr2=0x7fcbd0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 shutdown_connections 2024-09-19T05:40:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:19.767+0000 7fcbd5dfe640 1 -- 172.21.15.38:0/4288655947 wait complete. 2024-09-19T05:40:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 -- 172.21.15.38:0/2367546663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874102570 msgr2=0x7fc8741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 --2- 172.21.15.38:0/2367546663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874102570 0x7fc8741029b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc868009900 tx=0x7fc86802a480 comp rx=0 tx=0).stop 2024-09-19T05:40:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 -- 172.21.15.38:0/2367546663 shutdown_connections 2024-09-19T05:40:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 --2- 172.21.15.38:0/2367546663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc874102ef0 0x7fc87410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 --2- 172.21.15.38:0/2367546663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874102570 0x7fc8741029b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 --2- 172.21.15.38:0/2367546663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874100e20 0x7fc874101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 -- 172.21.15.38:0/2367546663 >> 172.21.15.38:0/2367546663 conn(0x7fc8740fc7d0 msgr2=0x7fc8740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:20.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 -- 172.21.15.38:0/2367546663 shutdown_connections 2024-09-19T05:40:20.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.796+0000 7fc878efa640 1 -- 172.21.15.38:0/2367546663 wait complete. 2024-09-19T05:40:20.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.797+0000 7fc878efa640 1 Processor -- start 2024-09-19T05:40:20.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.797+0000 7fc878efa640 1 -- start start 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874100e20 0x7fc874198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874102570 0x7fc874198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc874102ef0 0x7fc87419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc874110f80 con 0x7fc874100e20 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc874110e00 con 0x7fc874102570 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc878efa640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc874111100 con 0x7fc874102ef0 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc871d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874102570 0x7fc874198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874100e20 0x7fc874198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc871d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874102570 0x7fc874198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:56894/0 (socket says 172.21.15.38:56894) 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc871d9b640 1 -- 172.21.15.38:0/135394931 learned_addr learned my addr 172.21.15.38:0/135394931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc872d9d640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc874102ef0 0x7fc87419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 -- 172.21.15.38:0/135394931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc874102ef0 msgr2=0x7fc87419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc874102ef0 0x7fc87419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 -- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874102570 msgr2=0x7fc874198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc874102570 0x7fc874198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87419d3c0 con 0x7fc874100e20 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.798+0000 7fc87259c640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874100e20 0x7fc874198410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c970 tx=0x7fc86400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.799+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864007bb0 con 0x7fc874100e20 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.799+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc864007d50 con 0x7fc874100e20 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.799+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87419d6b0 con 0x7fc874100e20 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.799+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864018820 con 0x7fc874100e20 2024-09-19T05:40:20.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.799+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8741a4f90 con 0x7fc874100e20 2024-09-19T05:40:20.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.800+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc874062940 con 0x7fc874100e20 2024-09-19T05:40:20.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.801+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc8640189c0 con 0x7fc874100e20 2024-09-19T05:40:20.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc8637fe640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc85c0a32d0 0x7fc85c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc8640d0480 con 0x7fc874100e20 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc871d9b640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc85c0a32d0 0x7fc85c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc8637fe640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] conn(0x7fc85c0ae4a0 0x7fc85c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc864018e50 con 0x7fc85c0ae4a0 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc8640d0830 con 0x7fc874100e20 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.802+0000 7fc871d9b640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc85c0a32d0 0x7fc85c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc868005770 tx=0x7fc868005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.803+0000 7fc872d9d640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] conn(0x7fc85c0ae4a0 0x7fc85c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:20.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.803+0000 7fc872d9d640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] conn(0x7fc85c0ae4a0 0x7fc85c0b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:20.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.804+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== osd.7 v2:172.21.15.47:6816/2705740748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc864018e50 con 0x7fc85c0ae4a0 2024-09-19T05:40:20.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.821+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 --> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc874062dc0 con 0x7fc85c0ae4a0 2024-09-19T05:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.821+0000 7fc8637fe640 1 -- 172.21.15.38:0/135394931 <== osd.7 v2:172.21.15.47:6816/2705740748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc874062dc0 con 0x7fc85c0ae4a0 2024-09-19T05:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] conn(0x7fc85c0ae4a0 msgr2=0x7fc85c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] conn(0x7fc85c0ae4a0 0x7fc85c0b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc85c0a32d0 msgr2=0x7fc85c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc85c0a32d0 0x7fc85c0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc868005770 tx=0x7fc868005a20 comp rx=0 tx=0).stop 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874100e20 msgr2=0x7fc874198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 --2- 172.21.15.38:0/135394931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc874100e20 0x7fc874198410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c970 tx=0x7fc86400ce30 comp rx=0 tx=0).stop 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc872d9d640 1 -- 172.21.15.38:0/135394931 reap_dead start 2024-09-19T05:40:20.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 shutdown_connections 2024-09-19T05:40:20.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 >> 172.21.15.38:0/135394931 conn(0x7fc8740fc7d0 msgr2=0x7fc87406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:20.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 shutdown_connections 2024-09-19T05:40:20.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:20.822+0000 7fc878efa640 1 -- 172.21.15.38:0/135394931 wait complete. 2024-09-19T05:40:20.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:20 smithi038 ceph-mon[31227]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:40:21.028 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:20 smithi047 ceph-mon[35327]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:40:21.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:20 smithi179 ceph-mon[35781]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:40:21.294 INFO:teuthology.orchestra.run.smithi038.stdout:322122547227 2024-09-19T05:40:21.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T05:40:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- 172.21.15.38:0/1089223040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff770102da0 msgr2=0x7ff770103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1089223040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff770102da0 0x7ff770103180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff760009900 tx=0x7ff76002a480 comp rx=0 tx=0).stop 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- 172.21.15.38:0/1089223040 shutdown_connections 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1089223040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff77010c180 0x7ff77010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1089223040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7701036c0 0x7ff77010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1089223040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff770102da0 0x7ff770103180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- 172.21.15.38:0/1089223040 >> 172.21.15.38:0/1089223040 conn(0x7ff7700fc7a0 msgr2=0x7ff7700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- 172.21.15.38:0/1089223040 shutdown_connections 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- 172.21.15.38:0/1089223040 wait complete. 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 Processor -- start 2024-09-19T05:40:22.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.136+0000 7ff7770ef640 1 -- start start 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 0x7ff7701984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7701036c0 0x7ff770198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 0x7ff77019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff770111f60 con 0x7ff77010c180 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff770111de0 con 0x7ff770102da0 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff7770ef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701120e0 con 0x7ff7701036c0 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff774e8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 0x7ff7701984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 0x7ff77019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff774e8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 0x7ff7701984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:56914/0 (socket says 172.21.15.38:56914) 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff774e8b640 1 -- 172.21.15.38:0/1080139306 learned_addr learned my addr 172.21.15.38:0/1080139306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 0x7ff77019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50742/0 (socket says 172.21.15.38:50742) 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff76ffff640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7701036c0 0x7ff770198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 -- 172.21.15.38:0/1080139306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7701036c0 msgr2=0x7ff770198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7701036c0 0x7ff770198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 -- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 msgr2=0x7ff7701984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 0x7ff7701984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff77568c640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77019d470 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.137+0000 7ff774e8b640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff770102da0 0x7ff7701984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff77568c640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 0x7ff77019cd90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff76800e940 tx=0x7ff76800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768004d20 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff768005020 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77019d760 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7680116b0 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.138+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a5040 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.139+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff770062900 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.140+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff7680054d0 con 0x7ff77010c180 2024-09-19T05:40:22.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff76dffb640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7580a3350 0x7ff7580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff774e8b640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7580a3350 0x7ff7580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff7680d0180 con 0x7ff77010c180 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff76dffb640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] conn(0x7ff7580ae520 0x7ff7580b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff768014ec0 con 0x7ff7580ae520 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff7680d05d0 con 0x7ff77010c180 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.141+0000 7ff774e8b640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7580a3350 0x7ff7580a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7600096c0 tx=0x7ff760005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.142+0000 7ff76ffff640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] conn(0x7ff7580ae520 0x7ff7580b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.142+0000 7ff76ffff640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] conn(0x7ff7580ae520 0x7ff7580b09c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.143+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== osd.4 v2:172.21.15.47:6808/3118152095 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff768014ec0 con 0x7ff7580ae520 2024-09-19T05:40:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 --> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff770062d80 con 0x7ff7580ae520 2024-09-19T05:40:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff76dffb640 1 -- 172.21.15.38:0/1080139306 <== osd.4 v2:172.21.15.47:6808/3118152095 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff770062d80 con 0x7ff7580ae520 2024-09-19T05:40:22.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] conn(0x7ff7580ae520 msgr2=0x7ff7580b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] conn(0x7ff7580ae520 0x7ff7580b09c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7580a3350 msgr2=0x7ff7580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff7580a3350 0x7ff7580a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7600096c0 tx=0x7ff760005a20 comp rx=0 tx=0).stop 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 msgr2=0x7ff77019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff7770ef640 1 --2- 172.21.15.38:0/1080139306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff77010c180 0x7ff77019cd90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff76800e940 tx=0x7ff76800ee00 comp rx=0 tx=0).stop 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.159+0000 7ff77568c640 1 -- 172.21.15.38:0/1080139306 reap_dead start 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.160+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 shutdown_connections 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.160+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 >> 172.21.15.38:0/1080139306 conn(0x7ff7700fc7a0 msgr2=0x7ff7700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.160+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 shutdown_connections 2024-09-19T05:40:22.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.160+0000 7ff7770ef640 1 -- 172.21.15.38:0/1080139306 wait complete. 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 -- 172.21.15.38:0/1615829572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a5810c170 msgr2=0x7f4a5810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 --2- 172.21.15.38:0/1615829572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a5810c170 0x7f4a5810e610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ca0 tx=0x7f4a5002a950 comp rx=0 tx=0).stop 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 -- 172.21.15.38:0/1615829572 shutdown_connections 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 --2- 172.21.15.38:0/1615829572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a5810c170 0x7f4a5810e610 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 --2- 172.21.15.38:0/1615829572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a58103660 0x7f4a5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 --2- 172.21.15.38:0/1615829572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a58102d40 0x7f4a58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 -- 172.21.15.38:0/1615829572 >> 172.21.15.38:0/1615829572 conn(0x7f4a580fc7a0 msgr2=0x7f4a580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 -- 172.21.15.38:0/1615829572 shutdown_connections 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.605+0000 7f4a5db64640 1 -- 172.21.15.38:0/1615829572 wait complete. 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 Processor -- start 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 -- start start 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 0x7f4a5819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a58103660 0x7f4a5819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a5810c170 0x7f4a5819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58111f30 con 0x7f4a58102d40 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58111db0 con 0x7f4a58103660 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.606+0000 7f4a5db64640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581120b0 con 0x7f4a5810c170 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 0x7f4a5819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 0x7f4a5819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50760/0 (socket says 172.21.15.38:50760) 2024-09-19T05:40:22.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 -- 172.21.15.38:0/877080952 learned_addr learned my addr 172.21.15.38:0/877080952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a4effd640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a58103660 0x7f4a5819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a57fff640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a5810c170 0x7f4a5819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 -- 172.21.15.38:0/877080952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a5810c170 msgr2=0x7f4a5819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a5810c170 0x7f4a5819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 -- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a58103660 msgr2=0x7f4a5819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a58103660 0x7f4a5819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5819f5c0 con 0x7f4a58102d40 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a57fff640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a5810c170 0x7f4a5819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a577fe640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 0x7f4a5819a610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d950 tx=0x7f4a4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48017030 con 0x7f4a58102d40 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5819f8b0 con 0x7f4a58102d40 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48004020 con 0x7f4a58102d40 2024-09-19T05:40:22.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a480047a0 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.607+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a7190 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.608+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a58102450 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.610+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a480041d0 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.610+0000 7f4a557fa640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a440a32d0 0x7f4a440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a4effd640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a440a32d0 0x7f4a440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4a480cff90 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a557fa640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] conn(0x7f4a440ae4a0 0x7f4a440b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a440b1010 con 0x7f4a440ae4a0 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4a480d03e0 con 0x7f4a58102d40 2024-09-19T05:40:22.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a4effd640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a440a32d0 0x7f4a440a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a580febe0 tx=0x7f4a40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.611+0000 7f4a57fff640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] conn(0x7f4a440ae4a0 0x7f4a440b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.612+0000 7f4a57fff640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] conn(0x7f4a440ae4a0 0x7f4a440b0940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.612+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== osd.10 v2:172.21.15.47:6824/1633858254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a440b1010 con 0x7f4a440ae4a0 2024-09-19T05:40:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.626+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 --> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a5806b070 con 0x7f4a440ae4a0 2024-09-19T05:40:22.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a557fa640 1 -- 172.21.15.38:0/877080952 <== osd.10 v2:172.21.15.47:6824/1633858254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a5806b070 con 0x7f4a440ae4a0 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] conn(0x7f4a440ae4a0 msgr2=0x7f4a440b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] conn(0x7f4a440ae4a0 0x7f4a440b0940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a440a32d0 msgr2=0x7f4a440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a440a32d0 0x7f4a440a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a580febe0 tx=0x7f4a40005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 msgr2=0x7f4a5819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:22.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 --2- 172.21.15.38:0/877080952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a58102d40 0x7f4a5819a610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d950 tx=0x7f4a4800de10 comp rx=0 tx=0).stop 2024-09-19T05:40:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a57fff640 1 -- 172.21.15.38:0/877080952 reap_dead start 2024-09-19T05:40:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 shutdown_connections 2024-09-19T05:40:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 >> 172.21.15.38:0/877080952 conn(0x7f4a580fc7a0 msgr2=0x7f4a5810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 shutdown_connections 2024-09-19T05:40:22.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:22.627+0000 7f4a5db64640 1 -- 172.21.15.38:0/877080952 wait complete. 2024-09-19T05:40:22.844 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:22 smithi038 ceph-mon[31227]: pgmap v325: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 650 B/s wr, 12 op/s 2024-09-19T05:40:23.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:22 smithi179 ceph-mon[35781]: pgmap v325: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 650 B/s wr, 12 op/s 2024-09-19T05:40:23.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:22 smithi047 ceph-mon[35327]: pgmap v325: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 650 B/s wr, 12 op/s 2024-09-19T05:40:23.276 INFO:teuthology.orchestra.run.smithi038.stdout:270582939687 2024-09-19T05:40:23.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T05:40:23.700 INFO:teuthology.orchestra.run.smithi038.stdout:137438953536 2024-09-19T05:40:23.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 -- 172.21.15.38:0/3191660665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8100e20 msgr2=0x7fb2f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 --2- 172.21.15.38:0/3191660665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8100e20 0x7fb2f8101200 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec02a490 comp rx=0 tx=0).stop 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 -- 172.21.15.38:0/3191660665 shutdown_connections 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 --2- 172.21.15.38:0/3191660665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 0x7fb2f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 --2- 172.21.15.38:0/3191660665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f8102570 0x7fb2f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 --2- 172.21.15.38:0/3191660665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8100e20 0x7fb2f8101200 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec02a490 comp rx=0 tx=0).stop 2024-09-19T05:40:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 -- 172.21.15.38:0/3191660665 >> 172.21.15.38:0/3191660665 conn(0x7fb2f80fc7d0 msgr2=0x7fb2f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:23.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 -- 172.21.15.38:0/3191660665 shutdown_connections 2024-09-19T05:40:23.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.893+0000 7fb2fee42640 1 -- 172.21.15.38:0/3191660665 wait complete. 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 Processor -- start 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 -- start start 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8102570 0x7fb2f81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 0x7fb2f8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f819cf80 0x7fb2f819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f8112140 con 0x7fb2f8102570 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f806ae40 con 0x7fb2f819cf80 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fee42640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f81122c0 con 0x7fb2f8102ef0 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fcbde640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8102570 0x7fb2f81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fcbde640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8102570 0x7fb2f81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50778/0 (socket says 172.21.15.38:50778) 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2f7fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 0x7fb2f8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.894+0000 7fb2fcbde640 1 -- 172.21.15.38:0/2297155597 learned_addr learned my addr 172.21.15.38:0/2297155597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2fd3df640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f819cf80 0x7fb2f819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 -- 172.21.15.38:0/2297155597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f819cf80 msgr2=0x7fb2f819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2f819cf80 0x7fb2f819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 -- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8102570 msgr2=0x7fb2f81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2f8102570 0x7fb2f81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f819da40 con 0x7fb2f8102ef0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f7fff640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 0x7fb2f8198bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400d6d0 tx=0x7fb2e400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4017070 con 0x7fb2f8102ef0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4009820 con 0x7fb2f8102ef0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f819dd30 con 0x7fb2f8102ef0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.895+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004d00 con 0x7fb2f8102ef0 2024-09-19T05:40:23.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.896+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f81a5560 con 0x7fb2f8102ef0 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.896+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2f8062980 con 0x7fb2f8102ef0 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.897+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb2e40099d0 con 0x7fb2f8102ef0 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2f5ffb640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb2e80a3350 0x7fb2e80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2fcbde640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb2e80a3350 0x7fb2e80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb2e40cfa60 con 0x7fb2f8102ef0 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2f5ffb640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] conn(0x7fb2e80ae520 0x7fb2e80b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2e4047ea0 con 0x7fb2e80ae520 2024-09-19T05:40:23.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.898+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb2e40cfe10 con 0x7fb2f8102ef0 2024-09-19T05:40:23.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.899+0000 7fb2fd3df640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] conn(0x7fb2e80ae520 0x7fb2e80b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:23.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.899+0000 7fb2fcbde640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb2e80a3350 0x7fb2e80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:23.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.899+0000 7fb2fd3df640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] conn(0x7fb2e80ae520 0x7fb2e80b09c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:23.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.900+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== osd.8 v2:172.21.15.179:6816/2844755056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2e4047ea0 con 0x7fb2e80ae520 2024-09-19T05:40:23.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.915+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 --> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2f8062e00 con 0x7fb2e80ae520 2024-09-19T05:40:23.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2f5ffb640 1 -- 172.21.15.38:0/2297155597 <== osd.8 v2:172.21.15.179:6816/2844755056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb2f8062e00 con 0x7fb2e80ae520 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] conn(0x7fb2e80ae520 msgr2=0x7fb2e80b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] conn(0x7fb2e80ae520 0x7fb2e80b09c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb2e80a3350 msgr2=0x7fb2e80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb2e80a3350 0x7fb2e80a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec0058d0 comp rx=0 tx=0).stop 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 msgr2=0x7fb2f8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:23.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 --2- 172.21.15.38:0/2297155597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2f8102ef0 0x7fb2f8198bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400d6d0 tx=0x7fb2e400db90 comp rx=0 tx=0).stop 2024-09-19T05:40:23.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fd3df640 1 -- 172.21.15.38:0/2297155597 reap_dead start 2024-09-19T05:40:23.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 shutdown_connections 2024-09-19T05:40:23.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.916+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 >> 172.21.15.38:0/2297155597 conn(0x7fb2f80fc7d0 msgr2=0x7fb2f80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:23.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.917+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 shutdown_connections 2024-09-19T05:40:23.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:23.917+0000 7fb2fee42640 1 -- 172.21.15.38:0/2297155597 wait complete. 2024-09-19T05:40:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:24.856 INFO:teuthology.orchestra.run.smithi038.stdout:292057776162 2024-09-19T05:40:24.857 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T05:40:25.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:24 smithi038 ceph-mon[31227]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:24 smithi179 ceph-mon[35781]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:25.216 INFO:teuthology.orchestra.run.smithi038.stdout:34359738452 2024-09-19T05:40:25.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T05:40:25.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:24 smithi047 ceph-mon[35327]: pgmap v326: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:25.873 INFO:teuthology.orchestra.run.smithi038.stdout:210453397553 2024-09-19T05:40:25.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T05:40:26.334 INFO:teuthology.orchestra.run.smithi038.stdout:158913790011 2024-09-19T05:40:26.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T05:40:26.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:26 smithi038 ceph-mon[31227]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 -- 172.21.15.38:0/1487555155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828102ef0 msgr2=0x7f982810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 --2- 172.21.15.38:0/1487555155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828102ef0 0x7f982810da40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9820009ca0 tx=0x7f982002a950 comp rx=0 tx=0).stop 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 -- 172.21.15.38:0/1487555155 shutdown_connections 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 --2- 172.21.15.38:0/1487555155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828102ef0 0x7f982810da40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 --2- 172.21.15.38:0/1487555155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 --2- 172.21.15.38:0/1487555155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828100e20 0x7f9828101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 -- 172.21.15.38:0/1487555155 >> 172.21.15.38:0/1487555155 conn(0x7f98280fc7d0 msgr2=0x7f98280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 -- 172.21.15.38:0/1487555155 shutdown_connections 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.923+0000 7f982f558640 1 -- 172.21.15.38:0/1487555155 wait complete. 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 Processor -- start 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 -- start start 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828102ef0 0x7f982819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9828110f80 con 0x7f9828100e20 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9828110e00 con 0x7f9828102570 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982f558640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111100 con 0x7f9828102ef0 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50796/0 (socket says 172.21.15.38:50796) 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 -- 172.21.15.38:0/2080567762 learned_addr learned my addr 172.21.15.38:0/2080567762 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982daf5640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828102ef0 0x7f982819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982caf3640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 -- 172.21.15.38:0/2080567762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828102ef0 msgr2=0x7f982819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828102ef0 0x7f982819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 -- 172.21.15.38:0/2080567762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 msgr2=0x7f98281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982d2f4640 1 -- 172.21.15.38:0/2080567762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982819d410 con 0x7f9828100e20 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.924+0000 7f982caf3640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f982d2f4640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d950 tx=0x7f981c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c017030 con 0x7f9828100e20 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c004020 con 0x7f9828100e20 2024-09-19T05:40:27.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982819d700 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0047a0 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.925+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281a4fe0 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.926+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828101200 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.927+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f981c0041d0 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.928+0000 7f981a7fc640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 0x7f98140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.928+0000 7f982caf3640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 0x7f98140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.928+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f981c0cf8d0 con 0x7f9828100e20 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.928+0000 7f982caf3640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 0x7f98140a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98280ff0e0 tx=0x7f9824005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:27.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:26.932+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f981c09c0d0 con 0x7f9828100e20 2024-09-19T05:40:27.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.108+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f982810a0d0 con 0x7f9828100e20 2024-09-19T05:40:27.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.108+0000 7f981a7fc640 1 -- 172.21.15.38:0/2080567762 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f981c09b9d0 con 0x7f9828100e20 2024-09-19T05:40:27.111 INFO:teuthology.orchestra.run.smithi038.stdout:107374182470 2024-09-19T05:40:27.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.110+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 msgr2=0x7f98140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.110+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 0x7f98140a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98280ff0e0 tx=0x7f9824005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:27.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.110+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 msgr2=0x7f9828198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.110+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d950 tx=0x7f981c00de10 comp rx=0 tx=0).stop 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 shutdown_connections 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f98140a32d0 0x7f98140a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9828102ef0 0x7f982819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9828102570 0x7f98281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 --2- 172.21.15.38:0/2080567762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9828100e20 0x7f9828198460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 >> 172.21.15.38:0/2080567762 conn(0x7f98280fc7d0 msgr2=0x7f982806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 shutdown_connections 2024-09-19T05:40:27.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.111+0000 7f982f558640 1 -- 172.21.15.38:0/2080567762 wait complete. 2024-09-19T05:40:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:26 smithi179 ceph-mon[35781]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:27.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:26 smithi047 ceph-mon[35327]: pgmap v327: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 -- 172.21.15.38:0/1088145045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8102790 msgr2=0x7feec81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1088145045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8102790 0x7feec81067e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc02a410 comp rx=0 tx=0).stop 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 -- 172.21.15.38:0/1088145045 shutdown_connections 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1088145045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8106f10 0x7feec810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1088145045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8102790 0x7feec81067e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1088145045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8101de0 0x7feec81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 -- 172.21.15.38:0/1088145045 >> 172.21.15.38:0/1088145045 conn(0x7feec80fd8c0 msgr2=0x7feec80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 -- 172.21.15.38:0/1088145045 shutdown_connections 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.374+0000 7feecdfd1640 1 -- 172.21.15.38:0/1088145045 wait complete. 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 Processor -- start 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 -- start start 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8101de0 0x7feec81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 0x7feec81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feec8111f70 con 0x7feec8106f10 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feec8111df0 con 0x7feec8101de0 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feecdfd1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feec81120f0 con 0x7feec8102790 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60472/0 (socket says 172.21.15.38:60472) 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.375+0000 7feec7fff640 1 -- 172.21.15.38:0/1741487458 learned_addr learned my addr 172.21.15.38:0/1741487458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec77fe640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8101de0 0x7feec81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec6ffd640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 0x7feec81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 -- 172.21.15.38:0/1741487458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 msgr2=0x7feec81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 0x7feec81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 -- 172.21.15.38:0/1741487458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8101de0 msgr2=0x7feec81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8101de0 0x7feec81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 -- 172.21.15.38:0/1741487458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec819d460 con 0x7feec8106f10 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec6ffd640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 0x7feec81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec7fff640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feec000eaa0 tx=0x7feec000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec000cdb0 con 0x7feec8106f10 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec0005020 con 0x7feec8106f10 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec819d750 con 0x7feec8106f10 2024-09-19T05:40:27.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00116b0 con 0x7feec8106f10 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.376+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81a5030 con 0x7feec8106f10 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.377+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec81021c0 con 0x7feec8106f10 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.379+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7feec0005560 con 0x7feec8106f10 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.379+0000 7feec4ff9640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 0x7feeb00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.379+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feec00d0470 con 0x7feec8106f10 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.379+0000 7feec77fe640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 0x7feeb00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.380+0000 7feec77fe640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 0x7feeb00a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009b60 tx=0x7feeb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:27.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.382+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feec0016030 con 0x7feec8106f10 2024-09-19T05:40:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.564+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feec810a1b0 con 0x7feec8106f10 2024-09-19T05:40:27.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.564+0000 7feec4ff9640 1 -- 172.21.15.38:0/1741487458 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feec009ccd0 con 0x7feec8106f10 2024-09-19T05:40:27.567 INFO:teuthology.orchestra.run.smithi038.stdout:77309411404 2024-09-19T05:40:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 msgr2=0x7feeb00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 0x7feeb00a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009b60 tx=0x7feeb8005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 msgr2=0x7feec819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feec000eaa0 tx=0x7feec000ef60 comp rx=0 tx=0).stop 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 shutdown_connections 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feeb00a32d0 0x7feeb00a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feec8106f10 0x7feec819cd80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec8102790 0x7feec81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 --2- 172.21.15.38:0/1741487458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8101de0 0x7feec81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 >> 172.21.15.38:0/1741487458 conn(0x7feec80fd8c0 msgr2=0x7feec80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 shutdown_connections 2024-09-19T05:40:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:27.566+0000 7feecdfd1640 1 -- 172.21.15.38:0/1741487458 wait complete. 2024-09-19T05:40:28.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:27 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2080567762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:40:28.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:27 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1741487458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:40:28.077 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182470 for osd.3 2024-09-19T05:40:28.077 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:27 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2080567762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:40:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:27 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1741487458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:40:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:27 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2080567762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:40:28.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:27 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1741487458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:40:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411404 for osd.2 2024-09-19T05:40:28.720 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:28.847 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.848 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.858 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.858 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.870 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:28.955 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:28 smithi038 ceph-mon[31227]: pgmap v328: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:29.112 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:28 smithi179 ceph-mon[35781]: pgmap v328: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:29.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:28 smithi047 ceph-mon[35327]: pgmap v328: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 -- 172.21.15.38:0/1852607857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98106f10 msgr2=0x7f0e9810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 --2- 172.21.15.38:0/1852607857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98106f10 0x7f0e9810ea90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009ca0 tx=0x7f0e9002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 -- 172.21.15.38:0/1852607857 shutdown_connections 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 --2- 172.21.15.38:0/1852607857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98106f10 0x7f0e9810ea90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 --2- 172.21.15.38:0/1852607857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98102790 0x7f0e981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 --2- 172.21.15.38:0/1852607857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 0x7f0e981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 -- 172.21.15.38:0/1852607857 >> 172.21.15.38:0/1852607857 conn(0x7f0e980fd8c0 msgr2=0x7f0e980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 -- 172.21.15.38:0/1852607857 shutdown_connections 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.895+0000 7f0ea0611640 1 -- 172.21.15.38:0/1852607857 wait complete. 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.896+0000 7f0ea0611640 1 Processor -- start 2024-09-19T05:40:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.896+0000 7f0ea0611640 1 -- start start 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 0x7f0e98198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98106f10 0x7f0e9819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98111fc0 con 0x7f0e98102790 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98111e40 con 0x7f0e98101de0 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0ea0611640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98112140 con 0x7f0e98106f10 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60506/0 (socket says 172.21.15.38:60506) 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 -- 172.21.15.38:0/2689934549 learned_addr learned my addr 172.21.15.38:0/2689934549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9ebae640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98106f10 0x7f0e9819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9e3ad640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 0x7f0e98198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 -- 172.21.15.38:0/2689934549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98106f10 msgr2=0x7f0e9819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98106f10 0x7f0e9819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 -- 172.21.15.38:0/2689934549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 msgr2=0x7f0e98198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 0x7f0e98198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.897+0000 7f0e9dbac640 1 -- 172.21.15.38:0/2689934549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9819d410 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0e9dbac640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9f0 tx=0x7f0e8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800cd80 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e88005500 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88010690 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9819d700 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.898+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e98074420 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.899+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e981021c0 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.901+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0e880046c0 con 0x7f0e98102790 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.901+0000 7f0e8f7fe640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 0x7f0e840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:29.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.901+0000 7f0e9e3ad640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 0x7f0e840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:29.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.901+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0e880d0150 con 0x7f0e98102790 2024-09-19T05:40:29.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.902+0000 7f0e9e3ad640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 0x7f0e840a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e94010d80 tx=0x7f0e94012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:29.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.910+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e8809c9b0 con 0x7f0e98102790 2024-09-19T05:40:30.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.088+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0e9810a150 con 0x7f0e98102790 2024-09-19T05:40:30.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.089+0000 7f0e8f7fe640 1 -- 172.21.15.38:0/2689934549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e88096e20 con 0x7f0e98102790 2024-09-19T05:40:30.091 INFO:teuthology.orchestra.run.smithi038.stdout:34359738454 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 msgr2=0x7f0e840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 0x7f0e840a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e94010d80 tx=0x7f0e94012040 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 msgr2=0x7f0e981989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9f0 tx=0x7f0e8800edf0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 shutdown_connections 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0e840a3350 0x7f0e840a5810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e98106f10 0x7f0e9819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e98102790 0x7f0e981989a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 --2- 172.21.15.38:0/2689934549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e98101de0 0x7f0e98198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 >> 172.21.15.38:0/2689934549 conn(0x7f0e980fd8c0 msgr2=0x7f0e980feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 shutdown_connections 2024-09-19T05:40:30.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.091+0000 7f0ea0611640 1 -- 172.21.15.38:0/2689934549 wait complete. 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 -- 172.21.15.38:0/4120525278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 msgr2=0x7f9f8c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 --2- 172.21.15.38:0/4120525278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c103180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009900 tx=0x7f9f8802a480 comp rx=0 tx=0).stop 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 -- 172.21.15.38:0/4120525278 shutdown_connections 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 --2- 172.21.15.38:0/4120525278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 0x7f9f8c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 --2- 172.21.15.38:0/4120525278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 0x7f9f8c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 --2- 172.21.15.38:0/4120525278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c103180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 -- 172.21.15.38:0/4120525278 >> 172.21.15.38:0/4120525278 conn(0x7f9f8c0fc7a0 msgr2=0x7f9f8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 -- 172.21.15.38:0/4120525278 shutdown_connections 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.759+0000 7f9f93308640 1 -- 172.21.15.38:0/4120525278 wait complete. 2024-09-19T05:40:30.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.760+0000 7f9f93308640 1 Processor -- start 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.760+0000 7f9f93308640 1 -- start start 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 0x7f9f8c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 0x7f9f8c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c111f60 con 0x7f9f8c102da0 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c111de0 con 0x7f9f8c1036c0 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f93308640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c1120e0 con 0x7f9f8c10c180 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60482/0 (socket says 172.21.15.38:60482) 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 -- 172.21.15.38:0/1685255446 learned_addr learned my addr 172.21.15.38:0/1685255446 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f908a3640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 0x7f9f8c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f918a5640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 0x7f9f8c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 -- 172.21.15.38:0/1685255446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 msgr2=0x7f9f8c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 0x7f9f8c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 -- 172.21.15.38:0/1685255446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 msgr2=0x7f9f8c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 0x7f9f8c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 -- 172.21.15.38:0/1685255446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c19d3d0 con 0x7f9f8c102da0 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f910a4640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f88006010 tx=0x7f9f880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stdout:55834574930 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8803f030 con 0x7f9f8c102da0 2024-09-19T05:40:30.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.761+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f88003ce0 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.762+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c19d660 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.762+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88035070 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.762+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c1a4fa0 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.763+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c103180 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.764+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9f88042030 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.766+0000 7f9f827fc640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 0x7f9f780a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.766+0000 7f9f908a3640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 0x7f9f780a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.766+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9f880f0340 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.766+0000 7f9f908a3640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 0x7f9f780a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0095f0 tx=0x7f9f7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.770+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f880bcc60 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.957+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9f8c107220 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.957+0000 7f9f827fc640 1 -- 172.21.15.38:0/1685255446 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f880bc560 con 0x7f9f8c102da0 2024-09-19T05:40:30.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 msgr2=0x7f9f780a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 0x7f9f780a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0095f0 tx=0x7f9f7c005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 msgr2=0x7f9f8c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f88006010 tx=0x7f9f880042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 shutdown_connections 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9f780a3350 0x7f9f780a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f8c10c180 0x7f9f8c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f8c1036c0 0x7f9f8c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 --2- 172.21.15.38:0/1685255446 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f8c102da0 0x7f9f8c198420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 >> 172.21.15.38:0/1685255446 conn(0x7f9f8c0fc7a0 msgr2=0x7f9f8c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 shutdown_connections 2024-09-19T05:40:30.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:29.960+0000 7f9f93308640 1 -- 172.21.15.38:0/1685255446 wait complete. 2024-09-19T05:40:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 -- 172.21.15.38:0/626198705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409410c170 msgr2=0x7f409410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 --2- 172.21.15.38:0/626198705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409410c170 0x7f409410e610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f408c009ca0 tx=0x7f408c02a950 comp rx=0 tx=0).stop 2024-09-19T05:40:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 -- 172.21.15.38:0/626198705 shutdown_connections 2024-09-19T05:40:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 --2- 172.21.15.38:0/626198705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f409410c170 0x7f409410e610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 --2- 172.21.15.38:0/626198705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 0x7f409410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 --2- 172.21.15.38:0/626198705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4094102d40 0x7f4094103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 -- 172.21.15.38:0/626198705 >> 172.21.15.38:0/626198705 conn(0x7f40940fc7a0 msgr2=0x7f40940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 -- 172.21.15.38:0/626198705 shutdown_connections 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.899+0000 7f409bfbe640 1 -- 172.21.15.38:0/626198705 wait complete. 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 Processor -- start 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 -- start start 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 0x7f40941988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 0x7f409419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4094111f70 con 0x7f4094102d40 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4094111df0 con 0x7f4094103660 2024-09-19T05:40:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.900+0000 7f409bfbe640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40941120f0 con 0x7f409410c170 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60522/0 (socket says 172.21.15.38:60522) 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 -- 172.21.15.38:0/3208409823 learned_addr learned my addr 172.21.15.38:0/3208409823 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f409a55b640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 0x7f409419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099559640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 0x7f40941988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 -- 172.21.15.38:0/3208409823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 msgr2=0x7f409419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 0x7f409419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 -- 172.21.15.38:0/3208409823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 msgr2=0x7f40941988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 0x7f40941988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 -- 172.21.15.38:0/3208409823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419d320 con 0x7f4094102d40 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f409a55b640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 0x7f409419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f4099d5a640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f409000c890 tx=0x7f409000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090004220 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40900043c0 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409419d610 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900049e0 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.901+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a4ef0 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.902+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4094103120 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.904+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f40900079b0 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.904+0000 7f408affd640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 0x7f40800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.904+0000 7f4099559640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 0x7f40800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.904+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f40900cfeb0 con 0x7f4094102d40 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.905+0000 7f4099559640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 0x7f40800a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4084005fd0 tx=0x7f4084005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:30.907+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4090014030 con 0x7f4094102d40 2024-09-19T05:40:31.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.089+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f40941996d0 con 0x7f4094102d40 2024-09-19T05:40:31.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:30 smithi038 ceph-mon[31227]: pgmap v329: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:31.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:30 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1685255446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:40:31.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:30 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2689934549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:40:31.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.089+0000 7f408affd640 1 -- 172.21.15.38:0/3208409823 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f409009c790 con 0x7f4094102d40 2024-09-19T05:40:31.092 INFO:teuthology.orchestra.run.smithi038.stdout:210453397554 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 msgr2=0x7f40800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 0x7f40800a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4084005fd0 tx=0x7f4084005f60 comp rx=0 tx=0).stop 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 msgr2=0x7f4094198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f409000c890 tx=0x7f409000cd50 comp rx=0 tx=0).stop 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 shutdown_connections 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40800a3350 0x7f40800a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:31.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409410c170 0x7f409419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094103660 0x7f40941988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 --2- 172.21.15.38:0/3208409823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4094102d40 0x7f4094198370 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 >> 172.21.15.38:0/3208409823 conn(0x7f40940fc7a0 msgr2=0x7f40940fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 shutdown_connections 2024-09-19T05:40:31.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:31.091+0000 7f409bfbe640 1 -- 172.21.15.38:0/3208409823 wait complete. 2024-09-19T05:40:31.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:30 smithi179 ceph-mon[35781]: pgmap v329: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:31.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:30 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1685255446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:40:31.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:30 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2689934549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:40:31.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:30 smithi047 ceph-mon[35327]: pgmap v329: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:40:31.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:30 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1685255446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:40:31.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:30 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2689934549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:40:31.419 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574930 for osd.1 2024-09-19T05:40:31.419 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:31.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738454 for osd.0 2024-09-19T05:40:31.840 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:32.055 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3208409823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:40:32.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:31 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3208409823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:40:32.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:31 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3208409823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:40:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 -- 172.21.15.38:0/2488325830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1082a0 msgr2=0x7f647c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 --2- 172.21.15.38:0/2488325830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1082a0 0x7f647c10ec40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6474009930 tx=0x7f647402a420 comp rx=0 tx=0).stop 2024-09-19T05:40:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 -- 172.21.15.38:0/2488325830 shutdown_connections 2024-09-19T05:40:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 --2- 172.21.15.38:0/2488325830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1082a0 0x7f647c10ec40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 --2- 172.21.15.38:0/2488325830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 0x7f647c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 --2- 172.21.15.38:0/2488325830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c102d30 0x7f647c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 -- 172.21.15.38:0/2488325830 >> 172.21.15.38:0/2488325830 conn(0x7f647c0fc7d0 msgr2=0x7f647c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.245+0000 7f64819ce640 1 -- 172.21.15.38:0/2488325830 shutdown_connections 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 -- 172.21.15.38:0/2488325830 wait complete. 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 Processor -- start 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 -- start start 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 0x7f647c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c1082a0 0x7f647c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1120a0 con 0x7f647c102d30 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f647c111f20 con 0x7f647c1036e0 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.246+0000 7f64819ce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f647c112220 con 0x7f647c1082a0 2024-09-19T05:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60536/0 (socket says 172.21.15.38:60536) 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 -- 172.21.15.38:0/888388919 learned_addr learned my addr 172.21.15.38:0/888388919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647a7fc640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 0x7f647c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647b7fe640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c1082a0 0x7f647c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 -- 172.21.15.38:0/888388919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c1082a0 msgr2=0x7f647c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c1082a0 0x7f647c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 -- 172.21.15.38:0/888388919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 msgr2=0x7f647c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 0x7f647c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 -- 172.21.15.38:0/888388919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c19d4e0 con 0x7f647c102d30 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f647affd640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f646c00eab0 tx=0x7f646c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00ce40 con 0x7f647c102d30 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c011950 con 0x7f647c102d30 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c19d7d0 con 0x7f647c102d30 2024-09-19T05:40:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c010ae0 con 0x7f647c102d30 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.247+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c1a50b0 con 0x7f647c102d30 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.249+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c103110 con 0x7f647c102d30 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.250+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f646c011b00 con 0x7f647c102d30 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.250+0000 7f64809cc640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 0x7f64640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.250+0000 7f647a7fc640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 0x7f64640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.250+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f646c0d0440 con 0x7f647c102d30 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.251+0000 7f647a7fc640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 0x7f64640a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f647c0fed80 tx=0x7f647000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.254+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f646c016030 con 0x7f647c102d30 2024-09-19T05:40:32.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.436+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f647c10a2b0 con 0x7f647c102d30 2024-09-19T05:40:32.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.436+0000 7f64809cc640 1 -- 172.21.15.38:0/888388919 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f646c09cca0 con 0x7f647c102d30 2024-09-19T05:40:32.438 INFO:teuthology.orchestra.run.smithi038.stdout:270582939690 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 msgr2=0x7f64640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 0x7f64640a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f647c0fed80 tx=0x7f647000b390 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 msgr2=0x7f647c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f646c00eab0 tx=0x7f646c00ef70 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 shutdown_connections 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f64640a32d0 0x7f64640a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c1082a0 0x7f647c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f647c1036e0 0x7f647c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 --2- 172.21.15.38:0/888388919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c102d30 0x7f647c198530 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 >> 172.21.15.38:0/888388919 conn(0x7f647c0fc7d0 msgr2=0x7f647c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 shutdown_connections 2024-09-19T05:40:32.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:32.439+0000 7f64819ce640 1 -- 172.21.15.38:0/888388919 wait complete. 2024-09-19T05:40:33.013 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:32 smithi038 ceph-mon[31227]: pgmap v330: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:33.013 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:32 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/888388919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:40:33.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:32 smithi179 ceph-mon[35781]: pgmap v330: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:33.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:32 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/888388919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:40:33.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:32 smithi047 ceph-mon[35327]: pgmap v330: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-19T05:40:33.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:32 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/888388919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:40:33.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 -- 172.21.15.38:0/3159546782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 msgr2=0x7efc3c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 --2- 172.21.15.38:0/3159546782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c1021c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc30009900 tx=0x7efc3002a410 comp rx=0 tx=0).stop 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 -- 172.21.15.38:0/3159546782 shutdown_connections 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 --2- 172.21.15.38:0/3159546782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c106f10 0x7efc3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 --2- 172.21.15.38:0/3159546782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c102790 0x7efc3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 --2- 172.21.15.38:0/3159546782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c1021c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 -- 172.21.15.38:0/3159546782 >> 172.21.15.38:0/3159546782 conn(0x7efc3c0fd8c0 msgr2=0x7efc3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 -- 172.21.15.38:0/3159546782 shutdown_connections 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.300+0000 7efc43f92640 1 -- 172.21.15.38:0/3159546782 wait complete. 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 Processor -- start 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 -- start start 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c102790 0x7efc3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 0x7efc3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c111f30 con 0x7efc3c101de0 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c111db0 con 0x7efc3c102790 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc43f92640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1120b0 con 0x7efc3c106f10 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc41d2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc41d2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60552/0 (socket says 172.21.15.38:60552) 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc41d2e640 1 -- 172.21.15.38:0/2702632599 learned_addr learned my addr 172.21.15.38:0/2702632599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc4152d640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c102790 0x7efc3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.301+0000 7efc4252f640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 0x7efc3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 -- 172.21.15.38:0/2702632599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 msgr2=0x7efc3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 0x7efc3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 -- 172.21.15.38:0/2702632599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c102790 msgr2=0x7efc3c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c102790 0x7efc3c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 -- 172.21.15.38:0/2702632599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a6920 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc4252f640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 0x7efc3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc41d2e640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3003f030 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc30003ce0 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30035070 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a6bb0 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.302+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a7090 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.303+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c1021c0 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.304+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efc30042030 con 0x7efc3c101de0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.305+0000 7efc2effd640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 0x7efc280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.305+0000 7efc4152d640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 0x7efc280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.305+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc300f0340 con 0x7efc3c101de0 2024-09-19T05:40:33.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.306+0000 7efc4152d640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 0x7efc280a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc3c075f80 tx=0x7efc3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:33.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.309+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc300bcd00 con 0x7efc3c101de0 2024-09-19T05:40:33.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.489+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efc3c109b60 con 0x7efc3c101de0 2024-09-19T05:40:33.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.489+0000 7efc2effd640 1 -- 172.21.15.38:0/2702632599 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc300bc600 con 0x7efc3c101de0 2024-09-19T05:40:33.491 INFO:teuthology.orchestra.run.smithi038.stdout:137438953538 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 msgr2=0x7efc280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 0x7efc280a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc3c075f80 tx=0x7efc3800b410 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 msgr2=0x7efc3c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc300042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 shutdown_connections 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efc280a32d0 0x7efc280a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc3c106f10 0x7efc3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc3c102790 0x7efc3c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 --2- 172.21.15.38:0/2702632599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc3c101de0 0x7efc3c0744f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 >> 172.21.15.38:0/2702632599 conn(0x7efc3c0fd8c0 msgr2=0x7efc3c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 shutdown_connections 2024-09-19T05:40:33.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.491+0000 7efc43f92640 1 -- 172.21.15.38:0/2702632599 wait complete. 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 -- 172.21.15.38:0/1905869422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 msgr2=0x7f3a281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 --2- 172.21.15.38:0/1905869422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281067e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02a410 comp rx=0 tx=0).stop 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 -- 172.21.15.38:0/1905869422 shutdown_connections 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 --2- 172.21.15.38:0/1905869422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 0x7f3a2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 --2- 172.21.15.38:0/1905869422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281067e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 --2- 172.21.15.38:0/1905869422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 0x7f3a281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 -- 172.21.15.38:0/1905869422 >> 172.21.15.38:0/1905869422 conn(0x7f3a280fd8c0 msgr2=0x7f3a280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:33.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 -- 172.21.15.38:0/1905869422 shutdown_connections 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.874+0000 7f3a30291640 1 -- 172.21.15.38:0/1905869422 wait complete. 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 Processor -- start 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 -- start start 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 0x7f3a28198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 0x7f3a2819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111f70 con 0x7f3a28102790 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111df0 con 0x7f3a28101de0 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a30291640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281120f0 con 0x7f3a28106f10 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a2d82c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a2d82c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60568/0 (socket says 172.21.15.38:60568) 2024-09-19T05:40:33.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a2d82c640 1 -- 172.21.15.38:0/1794779151 learned_addr learned my addr 172.21.15.38:0/1794779151 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.875+0000 7f3a2e82e640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 0x7f3a2819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2e02d640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 0x7f3a28198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 -- 172.21.15.38:0/1794779151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 msgr2=0x7f3a2819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 0x7f3a2819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 -- 172.21.15.38:0/1794779151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 msgr2=0x7f3a28198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 0x7f3a28198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 -- 172.21.15.38:0/1794779151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2819d420 con 0x7f3a28102790 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a2d82c640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c03f030 con 0x7f3a28102790 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c003ce0 con 0x7f3a28102790 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2819d6b0 con 0x7f3a28102790 2024-09-19T05:40:33.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c035070 con 0x7f3a28102790 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.876+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a281a4ff0 con 0x7f3a28102790 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.877+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a281032c0 con 0x7f3a28102790 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.878+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3a1c042030 con 0x7f3a28102790 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.879+0000 7f3a1b7fe640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 0x7f3a140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.879+0000 7f3a2e02d640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 0x7f3a140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.879+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3a1c0f00e0 con 0x7f3a28102790 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.879+0000 7f3a2e02d640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 0x7f3a140a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005fd0 tx=0x7f3a24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:33.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:33.883+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a1c0bca00 con 0x7f3a28102790 2024-09-19T05:40:34.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.060+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3a2810a150 con 0x7f3a28102790 2024-09-19T05:40:34.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.060+0000 7f3a1b7fe640 1 -- 172.21.15.38:0/1794779151 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3a1c0bc300 con 0x7f3a28102790 2024-09-19T05:40:34.062 INFO:teuthology.orchestra.run.smithi038.stdout:188978561081 2024-09-19T05:40:34.062 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:33 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2702632599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:40:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 msgr2=0x7f3a140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 0x7f3a140a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005fd0 tx=0x7f3a24005850 comp rx=0 tx=0).stop 2024-09-19T05:40:34.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 msgr2=0x7f3a281989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 shutdown_connections 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3a140a32d0 0x7f3a140a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a28106f10 0x7f3a2819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a28102790 0x7f3a281989b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.062+0000 7f3a30291640 1 --2- 172.21.15.38:0/1794779151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a28101de0 0x7f3a28198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.063+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 >> 172.21.15.38:0/1794779151 conn(0x7f3a280fd8c0 msgr2=0x7f3a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.063+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 shutdown_connections 2024-09-19T05:40:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.063+0000 7f3a30291640 1 -- 172.21.15.38:0/1794779151 wait complete. 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.086+0000 7f15ed2e4640 1 -- 172.21.15.38:0/2372751180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e810c180 msgr2=0x7f15e810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.086+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/2372751180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e810c180 0x7f15e810e620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009ca0 tx=0x7f15e002a950 comp rx=0 tx=0).stop 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 -- 172.21.15.38:0/2372751180 shutdown_connections 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/2372751180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e810c180 0x7f15e810e620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/2372751180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e8103670 0x7f15e810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/2372751180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8102d50 0x7f15e8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 -- 172.21.15.38:0/2372751180 >> 172.21.15.38:0/2372751180 conn(0x7f15e80fc7d0 msgr2=0x7f15e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 -- 172.21.15.38:0/2372751180 shutdown_connections 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 -- 172.21.15.38:0/2372751180 wait complete. 2024-09-19T05:40:34.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 Processor -- start 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.087+0000 7f15ed2e4640 1 -- start start 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 0x7f15e8100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e810c180 0x7f15e8101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8111f80 con 0x7f15e8102d50 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8111e00 con 0x7f15e810c180 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15ed2e4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8112100 con 0x7f15e8103670 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60592/0 (socket says 172.21.15.38:60592) 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 -- 172.21.15.38:0/3309029070 learned_addr learned my addr 172.21.15.38:0/3309029070 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e67fc640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 0x7f15e8100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:34.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e77fe640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e810c180 0x7f15e8101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 -- 172.21.15.38:0/3309029070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 msgr2=0x7f15e8100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 0x7f15e8100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 -- 172.21.15.38:0/3309029070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e810c180 msgr2=0x7f15e8101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e810c180 0x7f15e8101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 -- 172.21.15.38:0/3309029070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81a6900 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e67fc640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 0x7f15e8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.088+0000 7f15e6ffd640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15d800d770 tx=0x7f15d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.089+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8017070 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.089+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81a6bf0 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.089+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d8005500 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.089+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8004e00 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.089+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a7130 con 0x7f15e8102d50 2024-09-19T05:40:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.090+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e8103130 con 0x7f15e8102d50 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.091+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f15d8004fa0 con 0x7f15e8102d50 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.092+0000 7f15dffff640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 0x7f15d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.093+0000 7f15e67fc640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 0x7f15d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.093+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15d80cf830 con 0x7f15e8102d50 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.093+0000 7f15e67fc640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 0x7f15d40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15e8102440 tx=0x7f15d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:34.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.097+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15d809b9e0 con 0x7f15e8102d50 2024-09-19T05:40:34.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:33 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2702632599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:40:34.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:33 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2702632599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:40:34.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.277+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f15e8107220 con 0x7f15e8102d50 2024-09-19T05:40:34.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.277+0000 7f15dffff640 1 -- 172.21.15.38:0/3309029070 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f15d809fce0 con 0x7f15e8102d50 2024-09-19T05:40:34.279 INFO:teuthology.orchestra.run.smithi038.stdout:322122547230 2024-09-19T05:40:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 msgr2=0x7f15d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 0x7f15d40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15e8102440 tx=0x7f15d0005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 msgr2=0x7f15e81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15d800d770 tx=0x7f15d800dc30 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 shutdown_connections 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f15d40a32d0 0x7f15d40a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15e810c180 0x7f15e8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15e8103670 0x7f15e8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 --2- 172.21.15.38:0/3309029070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e8102d50 0x7f15e81009b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 >> 172.21.15.38:0/3309029070 conn(0x7f15e80fc7d0 msgr2=0x7f15e80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 shutdown_connections 2024-09-19T05:40:34.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:34.279+0000 7f15ed2e4640 1 -- 172.21.15.38:0/3309029070 wait complete. 2024-09-19T05:40:34.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561078 got 188978561081 for osd.6 2024-09-19T05:40:34.810 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:35.098 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:34 smithi038 ceph-mon[31227]: pgmap v331: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:35.098 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:34 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1794779151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:40:35.098 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:34 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3309029070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:40:35.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:34 smithi179 ceph-mon[35781]: pgmap v331: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:35.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:34 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1794779151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:40:35.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:34 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3309029070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:40:35.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:34 smithi047 ceph-mon[35327]: pgmap v331: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:35.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:34 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1794779151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:40:35.230 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:34 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3309029070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:40:35.527 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939687 got 270582939690 for osd.9 2024-09-19T05:40:35.527 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:35.652 INFO:teuthology.orchestra.run.smithi038.stdout:240518168621 2024-09-19T05:40:35.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T05:40:36.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953536 got 137438953538 for osd.4 2024-09-19T05:40:36.592 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:36.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.784+0000 7f7496d43640 1 -- 172.21.15.38:0/1257693066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 msgr2=0x7f749010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.784+0000 7f7496d43640 1 --2- 172.21.15.38:0/1257693066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749010e610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ca0 tx=0x7f748802a950 comp rx=0 tx=0).stop 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.784+0000 7f7496d43640 1 -- 172.21.15.38:0/1257693066 shutdown_connections 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.784+0000 7f7496d43640 1 --2- 172.21.15.38:0/1257693066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749010e610 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 --2- 172.21.15.38:0/1257693066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490103660 0x7f749010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 --2- 172.21.15.38:0/1257693066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490102d40 0x7f7490103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- 172.21.15.38:0/1257693066 >> 172.21.15.38:0/1257693066 conn(0x7f74900fc7a0 msgr2=0x7f74900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- 172.21.15.38:0/1257693066 shutdown_connections 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- 172.21.15.38:0/1257693066 wait complete. 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 Processor -- start 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- start start 2024-09-19T05:40:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 0x7f7490198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490103660 0x7f7490198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7490111f30 con 0x7f749010c170 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7490111db0 con 0x7f7490102d40 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.785+0000 7f7496d43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74901120b0 con 0x7f7490103660 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60608/0 (socket says 172.21.15.38:60608) 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 -- 172.21.15.38:0/1292321025 learned_addr learned my addr 172.21.15.38:0/1292321025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:36.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f748ffff640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490103660 0x7f7490198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f7494adf640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 0x7f7490198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 -- 172.21.15.38:0/1292321025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490103660 msgr2=0x7f7490198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490103660 0x7f7490198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 -- 172.21.15.38:0/1292321025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 msgr2=0x7f7490198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 0x7f7490198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 -- 172.21.15.38:0/1292321025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749019d3c0 con 0x7f749010c170 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f7494adf640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 0x7f7490198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f74952e0640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7488004af0 tx=0x7f748802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488034040 con 0x7f749010c170 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749019d650 con 0x7f749010c170 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488030920 con 0x7f749010c170 2024-09-19T05:40:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.786+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880047b0 con 0x7f749010c170 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.787+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a4f90 con 0x7f749010c170 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.788+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749006b070 con 0x7f749010c170 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.789+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7488037020 con 0x7f749010c170 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.789+0000 7f748dffb640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 0x7f74800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.790+0000 7f7494adf640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 0x7f74800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.790+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f74880f0b30 con 0x7f749010c170 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.790+0000 7f7494adf640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 0x7f74800a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74840058c0 tx=0x7f7484005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.794+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f74880bd440 con 0x7f749010c170 2024-09-19T05:40:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.976+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7490107210 con 0x7f749010c170 2024-09-19T05:40:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.976+0000 7f748dffb640 1 -- 172.21.15.38:0/1292321025 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f74880bcd40 con 0x7f749010c170 2024-09-19T05:40:36.978 INFO:teuthology.orchestra.run.smithi038.stdout:292057776164 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 msgr2=0x7f74800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 0x7f74800a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74840058c0 tx=0x7f7484005850 comp rx=0 tx=0).stop 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 msgr2=0x7f749019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7488004af0 tx=0x7f748802f5b0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 shutdown_connections 2024-09-19T05:40:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f74800a32d0 0x7f74800a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f749010c170 0x7f749019cce0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7490103660 0x7f7490198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 --2- 172.21.15.38:0/1292321025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7490102d40 0x7f7490198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 >> 172.21.15.38:0/1292321025 conn(0x7f74900fc7a0 msgr2=0x7f74900fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 shutdown_connections 2024-09-19T05:40:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:36.978+0000 7f7496d43640 1 -- 172.21.15.38:0/1292321025 wait complete. 2024-09-19T05:40:37.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:36 smithi038 ceph-mon[31227]: pgmap v332: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:40:37.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:36 smithi179 ceph-mon[35781]: pgmap v332: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:40:37.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:36 smithi047 ceph-mon[35327]: pgmap v332: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:40:37.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397553 got 210453397554 for osd.7 2024-09-19T05:40:37.647 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:37 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1292321025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:40:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:37 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2994350549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:40:38.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776164 for osd.10 2024-09-19T05:40:38.190 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:37 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1292321025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:40:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:37 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2994350549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:40:38.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:37 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1292321025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:40:38.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:37 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2994350549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:40:38.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.021+0000 7f07aecbf640 1 -- 172.21.15.38:0/96274560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8102570 msgr2=0x7f07a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.021+0000 7f07aecbf640 1 --2- 172.21.15.38:0/96274560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8102570 0x7f07a81029b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f079c009900 tx=0x7f079c02a480 comp rx=0 tx=0).stop 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- 172.21.15.38:0/96274560 shutdown_connections 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- 172.21.15.38:0/96274560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- 172.21.15.38:0/96274560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8102570 0x7f07a81029b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- 172.21.15.38:0/96274560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8100e20 0x7f07a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- 172.21.15.38:0/96274560 >> 172.21.15.38:0/96274560 conn(0x7f07a80fc7d0 msgr2=0x7f07a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- 172.21.15.38:0/96274560 shutdown_connections 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- 172.21.15.38:0/96274560 wait complete. 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 Processor -- start 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- start start 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:38.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 0x7f07a8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a81a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.022+0000 7f07aecbf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8110f80 con 0x7f07a8100e20 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aecbf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8110e00 con 0x7f07a8102570 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aecbf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8111100 con 0x7f07a8102ef0 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60624/0 (socket says 172.21.15.38:60624) 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 -- 172.21.15.38:0/2994350549 learned_addr learned my addr 172.21.15.38:0/2994350549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07a7fff640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 0x7f07a8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07ad25c640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a81a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 -- 172.21.15.38:0/2994350549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 msgr2=0x7f07a81a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a81a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 -- 172.21.15.38:0/2994350549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 msgr2=0x7f07a8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:38.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 0x7f07a8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 -- 172.21.15.38:0/2994350549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a81a47e0 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07a7fff640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 0x7f07a8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07ad25c640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a81a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aca5b640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f079800cce0 tx=0x7f079800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079800f650 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a81a49b0 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f079800fcd0 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.023+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798018c70 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.024+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a81a4ef0 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.024+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a8101200 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.026+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f079800f7f0 con 0x7f07a8100e20 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.026+0000 7f07a5ffb640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 0x7f07900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:38.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.026+0000 7f07a7fff640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 0x7f07900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.026+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07980d0370 con 0x7f07a8100e20 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.027+0000 7f07a7fff640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 0x7f07900a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c0041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.030+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0798014030 con 0x7f07a8100e20 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.213+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f07a806a560 con 0x7f07a8100e20 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stdout:158913790014 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.213+0000 7f07a5ffb640 1 -- 172.21.15.38:0/2994350549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f079809cbd0 con 0x7f07a8100e20 2024-09-19T05:40:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 msgr2=0x7f07900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 0x7f07900a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c0041e0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 msgr2=0x7f07a8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f079800cce0 tx=0x7f079800a620 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 shutdown_connections 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f07900a32d0 0x7f07900a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a8102ef0 0x7f07a81a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07a8102570 0x7f07a8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 --2- 172.21.15.38:0/2994350549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07a8100e20 0x7f07a8199220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 >> 172.21.15.38:0/2994350549 conn(0x7f07a80fc7d0 msgr2=0x7f07a8106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 shutdown_connections 2024-09-19T05:40:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:37.216+0000 7f07aecbf640 1 -- 172.21.15.38:0/2994350549 wait complete. 2024-09-19T05:40:38.932 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547227 got 322122547230 for osd.11 2024-09-19T05:40:38.933 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:39.116 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:38 smithi038 ceph-mon[31227]: pgmap v333: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T05:40:39.502 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:38 smithi047 ceph-mon[35327]: pgmap v333: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T05:40:39.502 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:38 smithi179 ceph-mon[35781]: pgmap v333: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-19T05:40:39.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790011 got 158913790014 for osd.5 2024-09-19T05:40:39.555 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:39.596 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 -- 172.21.15.38:0/4220040320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c102d30 msgr2=0x7fab7c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 --2- 172.21.15.38:0/4220040320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c102d30 0x7fab7c103110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009930 tx=0x7fab6c02a420 comp rx=0 tx=0).stop 2024-09-19T05:40:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 -- 172.21.15.38:0/4220040320 shutdown_connections 2024-09-19T05:40:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 --2- 172.21.15.38:0/4220040320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c1082a0 0x7fab7c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 --2- 172.21.15.38:0/4220040320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 0x7fab7c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 --2- 172.21.15.38:0/4220040320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c102d30 0x7fab7c103110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 -- 172.21.15.38:0/4220040320 >> 172.21.15.38:0/4220040320 conn(0x7fab7c0fc7d0 msgr2=0x7fab7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 -- 172.21.15.38:0/4220040320 shutdown_connections 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.104+0000 7fab80a90640 1 -- 172.21.15.38:0/4220040320 wait complete. 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 Processor -- start 2024-09-19T05:40:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 -- start start 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 0x7fab7c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 0x7fab7c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 0x7fab7c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c112100 con 0x7fab7c1082a0 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c111f80 con 0x7fab7c1036e0 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab80a90640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c112280 con 0x7fab7c102d30 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab7a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 0x7fab7c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab7ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 0x7fab7c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab7a59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 0x7fab7c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:59514/0 (socket says 172.21.15.38:59514) 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab7a59c640 1 -- 172.21.15.38:0/1389872548 learned_addr learned my addr 172.21.15.38:0/1389872548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.105+0000 7fab79d9b640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 0x7fab7c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 -- 172.21.15.38:0/1389872548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 msgr2=0x7fab7c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 0x7fab7c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 -- 172.21.15.38:0/1389872548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 msgr2=0x7fab7c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 0x7fab7c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 -- 172.21.15.38:0/1389872548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c19d5d0 con 0x7fab7c1082a0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab7ad9d640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 0x7fab7c19cef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab7400be20 tx=0x7fab7400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74019070 con 0x7fab7c1082a0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c19d8c0 con 0x7fab7c1082a0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab74014920 con 0x7fab7c1082a0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74012920 con 0x7fab7c1082a0 2024-09-19T05:40:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.106+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a51a0 con 0x7fab7c1082a0 2024-09-19T05:40:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.107+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c06a850 con 0x7fab7c1082a0 2024-09-19T05:40:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.109+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fab7401c070 con 0x7fab7c1082a0 2024-09-19T05:40:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.110+0000 7fab6b7fe640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 0x7fab640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:40.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.110+0000 7fab7a59c640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 0x7fab640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:40.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.110+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fab740cf620 con 0x7fab7c1082a0 2024-09-19T05:40:40.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.111+0000 7fab7a59c640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 0x7fab640a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab6c000c00 tx=0x7fab6c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:40.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.114+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab7409b8e0 con 0x7fab7c1082a0 2024-09-19T05:40:40.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.293+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fab7c10a2b0 con 0x7fab7c1082a0 2024-09-19T05:40:40.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.294+0000 7fab6b7fe640 1 -- 172.21.15.38:0/1389872548 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fab7409fbe0 con 0x7fab7c1082a0 2024-09-19T05:40:40.296 INFO:teuthology.orchestra.run.smithi038.stdout:240518168624 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 msgr2=0x7fab640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 0x7fab640a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab6c000c00 tx=0x7fab6c009560 comp rx=0 tx=0).stop 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 msgr2=0x7fab7c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 0x7fab7c19cef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab7400be20 tx=0x7fab7400bf10 comp rx=0 tx=0).stop 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 shutdown_connections 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fab640a32d0 0x7fab640a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab7c1082a0 0x7fab7c19cef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab7c1036e0 0x7fab7c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 --2- 172.21.15.38:0/1389872548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7c102d30 0x7fab7c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:40.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 >> 172.21.15.38:0/1389872548 conn(0x7fab7c0fc7d0 msgr2=0x7fab7c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:40.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 shutdown_connections 2024-09-19T05:40:40.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:40.296+0000 7fab80a90640 1 -- 172.21.15.38:0/1389872548 wait complete. 2024-09-19T05:40:41.032 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:40 smithi038 ceph-mon[31227]: pgmap v334: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:41.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:40 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1389872548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:40:41.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168621 got 240518168624 for osd.8 2024-09-19T05:40:41.034 DEBUG:teuthology.parallel:result is None 2024-09-19T05:40:41.034 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:40:41.035 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:40:41.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:40 smithi179 ceph-mon[35781]: pgmap v334: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:41.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:40 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1389872548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:40:41.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:40 smithi047 ceph-mon[35327]: pgmap v334: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:41.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:40 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1389872548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:40:41.297 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 -- 172.21.15.38:0/915590571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc308101280 msgr2=0x7fc30810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 --2- 172.21.15.38:0/915590571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc308101280 0x7fc30810df40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009900 tx=0x7fc2f802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 -- 172.21.15.38:0/915590571 shutdown_connections 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 --2- 172.21.15.38:0/915590571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc30810e480 0x7fc308110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 --2- 172.21.15.38:0/915590571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc308101280 0x7fc30810df40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 --2- 172.21.15.38:0/915590571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 0x7fc308100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 -- 172.21.15.38:0/915590571 >> 172.21.15.38:0/915590571 conn(0x7fc3080fc7a0 msgr2=0x7fc3080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:41.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 -- 172.21.15.38:0/915590571 shutdown_connections 2024-09-19T05:40:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 -- 172.21.15.38:0/915590571 wait complete. 2024-09-19T05:40:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.813+0000 7fc30fe56640 1 Processor -- start 2024-09-19T05:40:41.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 -- start start 2024-09-19T05:40:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 0x7fc30819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc308101280 0x7fc30819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081141a0 con 0x7fc30810e480 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc308114020 con 0x7fc3081008d0 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30fe56640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc308114320 con 0x7fc308101280 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42866/0 (socket says 172.21.15.38:42866) 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 -- 172.21.15.38:0/4006254255 learned_addr learned my addr 172.21.15.38:0/4006254255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30dbf2640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 0x7fc30819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30d3f1640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc308101280 0x7fc30819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 -- 172.21.15.38:0/4006254255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc308101280 msgr2=0x7fc30819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc308101280 0x7fc30819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 -- 172.21.15.38:0/4006254255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 msgr2=0x7fc30819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 0x7fc30819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.814+0000 7fc30e3f3640 1 -- 172.21.15.38:0/4006254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30819f6d0 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc30e3f3640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc300009fd0 tx=0x7fc30000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300004d20 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc300005020 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30819f9c0 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3000116d0 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.815+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a72a0 con 0x7fc30810e480 2024-09-19T05:40:41.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.816+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc308100cb0 con 0x7fc30810e480 2024-09-19T05:40:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.817+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc3000051d0 con 0x7fc30810e480 2024-09-19T05:40:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.818+0000 7fc2feffd640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 0x7fc2f40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.818+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc3000d01c0 con 0x7fc30810e480 2024-09-19T05:40:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.818+0000 7fc30dbf2640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 0x7fc2f40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:41.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.818+0000 7fc30dbf2640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 0x7fc2f40a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc30400bbe0 tx=0x7fc30400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:41.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:41.822+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc300016030 con 0x7fc30810e480 2024-09-19T05:40:42.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.001+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc308062630 con 0x7fc2f40a3350 2024-09-19T05:40:42.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.009+0000 7fc2feffd640 1 -- 172.21.15.38:0/4006254255 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7fc308062630 con 0x7fc2f40a3350 2024-09-19T05:40:42.014 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:40:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 msgr2=0x7fc2f40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:42.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 0x7fc2f40a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc30400bbe0 tx=0x7fc30400b390 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 msgr2=0x7fc30819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc300009fd0 tx=0x7fc30000ee60 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 shutdown_connections 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc2f40a3350 0x7fc2f40a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc30810e480 0x7fc30819eff0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc308101280 0x7fc30819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 --2- 172.21.15.38:0/4006254255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3081008d0 0x7fc30819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 >> 172.21.15.38:0/4006254255 conn(0x7fc3080fc7a0 msgr2=0x7fc3080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 shutdown_connections 2024-09-19T05:40:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:42.012+0000 7fc30fe56640 1 -- 172.21.15.38:0/4006254255 wait complete. 2024-09-19T05:40:42.694 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-19T05:40:41.608614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663218,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":696,"num_read_kb":479,"num_write":435,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":356,"ondisk_log_size":356,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355056,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586768,"statfs":{"total":1151940427776,"available":1151576850432,"internally_reserved":0,"allocated":14385152,"data_stored":7951496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003495"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:40.184549+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:40:40.184549+0000","last_peered":"2024-09-19T05:40:40.184549+0000","last_clean":"2024-09-19T05:40:40.184549+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:40:40.184549+0000","last_undegraded":"2024-09-19T05:40:40.184549+0000","last_fullsized":"2024-09-19T05:40:40.184549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593599+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:40:10.593599+0000","last_peered":"2024-09-19T05:40:10.593599+0000","last_clean":"2024-09-19T05:40:10.593599+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:40:10.593599+0000","last_undegraded":"2024-09-19T05:40:10.593599+0000","last_fullsized":"2024-09-19T05:40:10.593599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587091+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:40:10.587091+0000","last_peered":"2024-09-19T05:40:10.587091+0000","last_clean":"2024-09-19T05:40:10.587091+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:40:10.587091+0000","last_undegraded":"2024-09-19T05:40:10.587091+0000","last_fullsized":"2024-09-19T05:40:10.587091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593833+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:40:10.593833+0000","last_peered":"2024-09-19T05:40:10.593833+0000","last_clean":"2024-09-19T05:40:10.593833+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:40:10.593833+0000","last_undegraded":"2024-09-19T05:40:10.593833+0000","last_fullsized":"2024-09-19T05:40:10.593833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892882+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:40:10.892882+0000","last_peered":"2024-09-19T05:40:10.892882+0000","last_clean":"2024-09-19T05:40:10.892882+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:40:10.892882+0000","last_undegraded":"2024-09-19T05:40:10.892882+0000","last_fullsized":"2024-09-19T05:40:10.892882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588014+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:40:10.588014+0000","last_peered":"2024-09-19T05:40:10.588014+0000","last_clean":"2024-09-19T05:40:10.588014+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:40:10.588014+0000","last_undegraded":"2024-09-19T05:40:10.588014+0000","last_fullsized":"2024-09-19T05:40:10.588014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589582+0000","last_change":"2024-09-19T05:38:59.855714+0000","last_active":"2024-09-19T05:40:10.589582+0000","last_peered":"2024-09-19T05:40:10.589582+0000","last_clean":"2024-09-19T05:40:10.589582+0000","last_became_active":"2024-09-19T05:38:59.855268+0000","last_became_peered":"2024-09-19T05:38:59.855268+0000","last_unstale":"2024-09-19T05:40:10.589582+0000","last_undegraded":"2024-09-19T05:40:10.589582+0000","last_fullsized":"2024-09-19T05:40:10.589582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:20.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589637+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:40:10.589637+0000","last_peered":"2024-09-19T05:40:10.589637+0000","last_clean":"2024-09-19T05:40:10.589637+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:40:10.589637+0000","last_undegraded":"2024-09-19T05:40:10.589637+0000","last_fullsized":"2024-09-19T05:40:10.589637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595400+0000","last_change":"2024-09-19T05:40:10.595400+0000","last_active":"2024-09-19T05:40:10.595400+0000","last_peered":"2024-09-19T05:40:10.595400+0000","last_clean":"2024-09-19T05:40:10.595400+0000","last_became_active":"2024-09-19T05:40:10.595015+0000","last_became_peered":"2024-09-19T05:40:10.595015+0000","last_unstale":"2024-09-19T05:40:10.595400+0000","last_undegraded":"2024-09-19T05:40:10.595400+0000","last_fullsized":"2024-09-19T05:40:10.595400+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.449136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588735+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:40:10.588735+0000","last_peered":"2024-09-19T05:40:10.588735+0000","last_clean":"2024-09-19T05:40:10.588735+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:40:10.588735+0000","last_undegraded":"2024-09-19T05:40:10.588735+0000","last_fullsized":"2024-09-19T05:40:10.588735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591458+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:40:10.591458+0000","last_peered":"2024-09-19T05:40:10.591458+0000","last_clean":"2024-09-19T05:40:10.591458+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:40:10.591458+0000","last_undegraded":"2024-09-19T05:40:10.591458+0000","last_fullsized":"2024-09-19T05:40:10.591458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588329+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:40:10.588329+0000","last_peered":"2024-09-19T05:40:10.588329+0000","last_clean":"2024-09-19T05:40:10.588329+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:40:10.588329+0000","last_undegraded":"2024-09-19T05:40:10.588329+0000","last_fullsized":"2024-09-19T05:40:10.588329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587299+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:40:10.587299+0000","last_peered":"2024-09-19T05:40:10.587299+0000","last_clean":"2024-09-19T05:40:10.587299+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:40:10.587299+0000","last_undegraded":"2024-09-19T05:40:10.587299+0000","last_fullsized":"2024-09-19T05:40:10.587299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589368+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:40:10.589368+0000","last_peered":"2024-09-19T05:40:10.589368+0000","last_clean":"2024-09-19T05:40:10.589368+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:40:10.589368+0000","last_undegraded":"2024-09-19T05:40:10.589368+0000","last_fullsized":"2024-09-19T05:40:10.589368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588655+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:40:10.588655+0000","last_peered":"2024-09-19T05:40:10.588655+0000","last_clean":"2024-09-19T05:40:10.588655+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:40:10.588655+0000","last_undegraded":"2024-09-19T05:40:10.588655+0000","last_fullsized":"2024-09-19T05:40:10.588655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593712+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:40:10.593712+0000","last_peered":"2024-09-19T05:40:10.593712+0000","last_clean":"2024-09-19T05:40:10.593712+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:40:10.593712+0000","last_undegraded":"2024-09-19T05:40:10.593712+0000","last_fullsized":"2024-09-19T05:40:10.593712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588120+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:40:10.588120+0000","last_peered":"2024-09-19T05:40:10.588120+0000","last_clean":"2024-09-19T05:40:10.588120+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:40:10.588120+0000","last_undegraded":"2024-09-19T05:40:10.588120+0000","last_fullsized":"2024-09-19T05:40:10.588120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588698+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:40:10.588698+0000","last_peered":"2024-09-19T05:40:10.588698+0000","last_clean":"2024-09-19T05:40:10.588698+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:40:10.588698+0000","last_undegraded":"2024-09-19T05:40:10.588698+0000","last_fullsized":"2024-09-19T05:40:10.588698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594126+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:40:10.594126+0000","last_peered":"2024-09-19T05:40:10.594126+0000","last_clean":"2024-09-19T05:40:10.594126+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:40:10.594126+0000","last_undegraded":"2024-09-19T05:40:10.594126+0000","last_fullsized":"2024-09-19T05:40:10.594126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594555+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:40:10.594555+0000","last_peered":"2024-09-19T05:40:10.594555+0000","last_clean":"2024-09-19T05:40:10.594555+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:40:10.594555+0000","last_undegraded":"2024-09-19T05:40:10.594555+0000","last_fullsized":"2024-09-19T05:40:10.594555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585984+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:40:10.585984+0000","last_peered":"2024-09-19T05:40:10.585984+0000","last_clean":"2024-09-19T05:40:10.585984+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:40:10.585984+0000","last_undegraded":"2024-09-19T05:40:10.585984+0000","last_fullsized":"2024-09-19T05:40:10.585984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587945+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:40:10.587945+0000","last_peered":"2024-09-19T05:40:10.587945+0000","last_clean":"2024-09-19T05:40:10.587945+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:40:10.587945+0000","last_undegraded":"2024-09-19T05:40:10.587945+0000","last_fullsized":"2024-09-19T05:40:10.587945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589924+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:40:10.589924+0000","last_peered":"2024-09-19T05:40:10.589924+0000","last_clean":"2024-09-19T05:40:10.589924+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:40:10.589924+0000","last_undegraded":"2024-09-19T05:40:10.589924+0000","last_fullsized":"2024-09-19T05:40:10.589924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588877+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:40:10.588877+0000","last_peered":"2024-09-19T05:40:10.588877+0000","last_clean":"2024-09-19T05:40:10.588877+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:40:10.588877+0000","last_undegraded":"2024-09-19T05:40:10.588877+0000","last_fullsized":"2024-09-19T05:40:10.588877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892768+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:40:10.892768+0000","last_peered":"2024-09-19T05:40:10.892768+0000","last_clean":"2024-09-19T05:40:10.892768+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:40:10.892768+0000","last_undegraded":"2024-09-19T05:40:10.892768+0000","last_fullsized":"2024-09-19T05:40:10.892768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589464+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:40:10.589464+0000","last_peered":"2024-09-19T05:40:10.589464+0000","last_clean":"2024-09-19T05:40:10.589464+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:40:10.589464+0000","last_undegraded":"2024-09-19T05:40:10.589464+0000","last_fullsized":"2024-09-19T05:40:10.589464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587217+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:40:10.587217+0000","last_peered":"2024-09-19T05:40:10.587217+0000","last_clean":"2024-09-19T05:40:10.587217+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:40:10.587217+0000","last_undegraded":"2024-09-19T05:40:10.587217+0000","last_fullsized":"2024-09-19T05:40:10.587217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594349+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:40:10.594349+0000","last_peered":"2024-09-19T05:40:10.594349+0000","last_clean":"2024-09-19T05:40:10.594349+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:40:10.594349+0000","last_undegraded":"2024-09-19T05:40:10.594349+0000","last_fullsized":"2024-09-19T05:40:10.594349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586477+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:40:10.586477+0000","last_peered":"2024-09-19T05:40:10.586477+0000","last_clean":"2024-09-19T05:40:10.586477+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:40:10.586477+0000","last_undegraded":"2024-09-19T05:40:10.586477+0000","last_fullsized":"2024-09-19T05:40:10.586477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593449+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:40:10.593449+0000","last_peered":"2024-09-19T05:40:10.593449+0000","last_clean":"2024-09-19T05:40:10.593449+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:40:10.593449+0000","last_undegraded":"2024-09-19T05:40:10.593449+0000","last_fullsized":"2024-09-19T05:40:10.593449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595364+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:40:10.595364+0000","last_peered":"2024-09-19T05:40:10.595364+0000","last_clean":"2024-09-19T05:40:10.595364+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:40:10.595364+0000","last_undegraded":"2024-09-19T05:40:10.595364+0000","last_fullsized":"2024-09-19T05:40:10.595364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589625+0000","last_change":"2024-09-19T05:39:00.095972+0000","last_active":"2024-09-19T05:40:10.589625+0000","last_peered":"2024-09-19T05:40:10.589625+0000","last_clean":"2024-09-19T05:40:10.589625+0000","last_became_active":"2024-09-19T05:39:00.095558+0000","last_became_peered":"2024-09-19T05:39:00.095558+0000","last_unstale":"2024-09-19T05:40:10.589625+0000","last_undegraded":"2024-09-19T05:40:10.589625+0000","last_fullsized":"2024-09-19T05:40:10.589625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:13.708685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594520+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:40:10.594520+0000","last_peered":"2024-09-19T05:40:10.594520+0000","last_clean":"2024-09-19T05:40:10.594520+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:40:10.594520+0000","last_undegraded":"2024-09-19T05:40:10.594520+0000","last_fullsized":"2024-09-19T05:40:10.594520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588685+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:40:10.588685+0000","last_peered":"2024-09-19T05:40:10.588685+0000","last_clean":"2024-09-19T05:40:10.588685+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:40:10.588685+0000","last_undegraded":"2024-09-19T05:40:10.588685+0000","last_fullsized":"2024-09-19T05:40:10.588685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586324+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:40:10.586324+0000","last_peered":"2024-09-19T05:40:10.586324+0000","last_clean":"2024-09-19T05:40:10.586324+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:40:10.586324+0000","last_undegraded":"2024-09-19T05:40:10.586324+0000","last_fullsized":"2024-09-19T05:40:10.586324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587712+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:40:10.587712+0000","last_peered":"2024-09-19T05:40:10.587712+0000","last_clean":"2024-09-19T05:40:10.587712+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:40:10.587712+0000","last_undegraded":"2024-09-19T05:40:10.587712+0000","last_fullsized":"2024-09-19T05:40:10.587712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588398+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:40:10.588398+0000","last_peered":"2024-09-19T05:40:10.588398+0000","last_clean":"2024-09-19T05:40:10.588398+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:40:10.588398+0000","last_undegraded":"2024-09-19T05:40:10.588398+0000","last_fullsized":"2024-09-19T05:40:10.588398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:38.182517+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:40:38.182517+0000","last_peered":"2024-09-19T05:40:38.182517+0000","last_clean":"2024-09-19T05:40:38.182517+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:40:38.182517+0000","last_undegraded":"2024-09-19T05:40:38.182517+0000","last_fullsized":"2024-09-19T05:40:38.182517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595102+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:40:10.595102+0000","last_peered":"2024-09-19T05:40:10.595102+0000","last_clean":"2024-09-19T05:40:10.595102+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:40:10.595102+0000","last_undegraded":"2024-09-19T05:40:10.595102+0000","last_fullsized":"2024-09-19T05:40:10.595102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594007+0000","last_change":"2024-09-19T05:40:10.594007+0000","last_active":"2024-09-19T05:40:10.594007+0000","last_peered":"2024-09-19T05:40:10.594007+0000","last_clean":"2024-09-19T05:40:10.594007+0000","last_became_active":"2024-09-19T05:40:10.593493+0000","last_became_peered":"2024-09-19T05:40:10.593493+0000","last_unstale":"2024-09-19T05:40:10.594007+0000","last_undegraded":"2024-09-19T05:40:10.594007+0000","last_fullsized":"2024-09-19T05:40:10.594007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:58.225840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:38.181872+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:40:38.181872+0000","last_peered":"2024-09-19T05:40:38.181872+0000","last_clean":"2024-09-19T05:40:38.181872+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:40:38.181872+0000","last_undegraded":"2024-09-19T05:40:38.181872+0000","last_fullsized":"2024-09-19T05:40:38.181872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594940+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:40:10.594940+0000","last_peered":"2024-09-19T05:40:10.594940+0000","last_clean":"2024-09-19T05:40:10.594940+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:40:10.594940+0000","last_undegraded":"2024-09-19T05:40:10.594940+0000","last_fullsized":"2024-09-19T05:40:10.594940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594704+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:40:10.594704+0000","last_peered":"2024-09-19T05:40:10.594704+0000","last_clean":"2024-09-19T05:40:10.594704+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:40:10.594704+0000","last_undegraded":"2024-09-19T05:40:10.594704+0000","last_fullsized":"2024-09-19T05:40:10.594704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587062+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:40:10.587062+0000","last_peered":"2024-09-19T05:40:10.587062+0000","last_clean":"2024-09-19T05:40:10.587062+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:40:10.587062+0000","last_undegraded":"2024-09-19T05:40:10.587062+0000","last_fullsized":"2024-09-19T05:40:10.587062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592345+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:40:10.592345+0000","last_peered":"2024-09-19T05:40:10.592345+0000","last_clean":"2024-09-19T05:40:10.592345+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:40:10.592345+0000","last_undegraded":"2024-09-19T05:40:10.592345+0000","last_fullsized":"2024-09-19T05:40:10.592345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593560+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:40:10.593560+0000","last_peered":"2024-09-19T05:40:10.593560+0000","last_clean":"2024-09-19T05:40:10.593560+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:40:10.593560+0000","last_undegraded":"2024-09-19T05:40:10.593560+0000","last_fullsized":"2024-09-19T05:40:10.593560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590772+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:40:10.590772+0000","last_peered":"2024-09-19T05:40:10.590772+0000","last_clean":"2024-09-19T05:40:10.590772+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:40:10.590772+0000","last_undegraded":"2024-09-19T05:40:10.590772+0000","last_fullsized":"2024-09-19T05:40:10.590772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587267+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:40:10.587267+0000","last_peered":"2024-09-19T05:40:10.587267+0000","last_clean":"2024-09-19T05:40:10.587267+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:40:10.587267+0000","last_undegraded":"2024-09-19T05:40:10.587267+0000","last_fullsized":"2024-09-19T05:40:10.587267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590512+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:40:10.590512+0000","last_peered":"2024-09-19T05:40:10.590512+0000","last_clean":"2024-09-19T05:40:10.590512+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:40:10.590512+0000","last_undegraded":"2024-09-19T05:40:10.590512+0000","last_fullsized":"2024-09-19T05:40:10.590512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591099+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:40:10.591099+0000","last_peered":"2024-09-19T05:40:10.591099+0000","last_clean":"2024-09-19T05:40:10.591099+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:40:10.591099+0000","last_undegraded":"2024-09-19T05:40:10.591099+0000","last_fullsized":"2024-09-19T05:40:10.591099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:37.200103+0000","last_change":"2024-09-19T05:39:00.097011+0000","last_active":"2024-09-19T05:40:37.200103+0000","last_peered":"2024-09-19T05:40:37.200103+0000","last_clean":"2024-09-19T05:40:37.200103+0000","last_became_active":"2024-09-19T05:39:00.096349+0000","last_became_peered":"2024-09-19T05:39:00.096349+0000","last_unstale":"2024-09-19T05:40:37.200103+0000","last_undegraded":"2024-09-19T05:40:37.200103+0000","last_fullsized":"2024-09-19T05:40:37.200103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:45.484368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596589+0000","last_change":"2024-09-19T05:40:10.596589+0000","last_active":"2024-09-19T05:40:10.596589+0000","last_peered":"2024-09-19T05:40:10.596589+0000","last_clean":"2024-09-19T05:40:10.596589+0000","last_became_active":"2024-09-19T05:40:10.596087+0000","last_became_peered":"2024-09-19T05:40:10.596087+0000","last_unstale":"2024-09-19T05:40:10.596589+0000","last_undegraded":"2024-09-19T05:40:10.596589+0000","last_fullsized":"2024-09-19T05:40:10.596589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594372+0000","last_change":"2024-09-19T05:40:10.594372+0000","last_active":"2024-09-19T05:40:10.594372+0000","last_peered":"2024-09-19T05:40:10.594372+0000","last_clean":"2024-09-19T05:40:10.594372+0000","last_became_active":"2024-09-19T05:40:10.594018+0000","last_became_peered":"2024-09-19T05:40:10.594018+0000","last_unstale":"2024-09-19T05:40:10.594372+0000","last_undegraded":"2024-09-19T05:40:10.594372+0000","last_fullsized":"2024-09-19T05:40:10.594372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:23.706247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592489+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:40:10.592489+0000","last_peered":"2024-09-19T05:40:10.592489+0000","last_clean":"2024-09-19T05:40:10.592489+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:40:10.592489+0000","last_undegraded":"2024-09-19T05:40:10.592489+0000","last_fullsized":"2024-09-19T05:40:10.592489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892210+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:40:10.892210+0000","last_peered":"2024-09-19T05:40:10.892210+0000","last_clean":"2024-09-19T05:40:10.892210+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:40:10.892210+0000","last_undegraded":"2024-09-19T05:40:10.892210+0000","last_fullsized":"2024-09-19T05:40:10.892210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590878+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:40:10.590878+0000","last_peered":"2024-09-19T05:40:10.590878+0000","last_clean":"2024-09-19T05:40:10.590878+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:40:10.590878+0000","last_undegraded":"2024-09-19T05:40:10.590878+0000","last_fullsized":"2024-09-19T05:40:10.590878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595364+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:40:10.595364+0000","last_peered":"2024-09-19T05:40:10.595364+0000","last_clean":"2024-09-19T05:40:10.595364+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:40:10.595364+0000","last_undegraded":"2024-09-19T05:40:10.595364+0000","last_fullsized":"2024-09-19T05:40:10.595364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590770+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:40:10.590770+0000","last_peered":"2024-09-19T05:40:10.590770+0000","last_clean":"2024-09-19T05:40:10.590770+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:40:10.590770+0000","last_undegraded":"2024-09-19T05:40:10.590770+0000","last_fullsized":"2024-09-19T05:40:10.590770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594949+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:40:10.594949+0000","last_peered":"2024-09-19T05:40:10.594949+0000","last_clean":"2024-09-19T05:40:10.594949+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:40:10.594949+0000","last_undegraded":"2024-09-19T05:40:10.594949+0000","last_fullsized":"2024-09-19T05:40:10.594949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589482+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:40:10.589482+0000","last_peered":"2024-09-19T05:40:10.589482+0000","last_clean":"2024-09-19T05:40:10.589482+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:40:10.589482+0000","last_undegraded":"2024-09-19T05:40:10.589482+0000","last_fullsized":"2024-09-19T05:40:10.589482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590244+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:40:10.590244+0000","last_peered":"2024-09-19T05:40:10.590244+0000","last_clean":"2024-09-19T05:40:10.590244+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:40:10.590244+0000","last_undegraded":"2024-09-19T05:40:10.590244+0000","last_fullsized":"2024-09-19T05:40:10.590244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596504+0000","last_change":"2024-09-19T05:40:10.596504+0000","last_active":"2024-09-19T05:40:10.596504+0000","last_peered":"2024-09-19T05:40:10.596504+0000","last_clean":"2024-09-19T05:40:10.596504+0000","last_became_active":"2024-09-19T05:40:10.596112+0000","last_became_peered":"2024-09-19T05:40:10.596112+0000","last_unstale":"2024-09-19T05:40:10.596504+0000","last_undegraded":"2024-09-19T05:40:10.596504+0000","last_fullsized":"2024-09-19T05:40:10.596504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:02.818567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587791+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:40:10.587791+0000","last_peered":"2024-09-19T05:40:10.587791+0000","last_clean":"2024-09-19T05:40:10.587791+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:40:10.587791+0000","last_undegraded":"2024-09-19T05:40:10.587791+0000","last_fullsized":"2024-09-19T05:40:10.587791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595308+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:40:10.595308+0000","last_peered":"2024-09-19T05:40:10.595308+0000","last_clean":"2024-09-19T05:40:10.595308+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:40:10.595308+0000","last_undegraded":"2024-09-19T05:40:10.595308+0000","last_fullsized":"2024-09-19T05:40:10.595308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586964+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:40:10.586964+0000","last_peered":"2024-09-19T05:40:10.586964+0000","last_clean":"2024-09-19T05:40:10.586964+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:40:10.586964+0000","last_undegraded":"2024-09-19T05:40:10.586964+0000","last_fullsized":"2024-09-19T05:40:10.586964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:37.181105+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:40:37.181105+0000","last_peered":"2024-09-19T05:40:37.181105+0000","last_clean":"2024-09-19T05:40:37.181105+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:40:37.181105+0000","last_undegraded":"2024-09-19T05:40:37.181105+0000","last_fullsized":"2024-09-19T05:40:37.181105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586098+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:40:10.586098+0000","last_peered":"2024-09-19T05:40:10.586098+0000","last_clean":"2024-09-19T05:40:10.586098+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:40:10.586098+0000","last_undegraded":"2024-09-19T05:40:10.586098+0000","last_fullsized":"2024-09-19T05:40:10.586098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590903+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:40:10.590903+0000","last_peered":"2024-09-19T05:40:10.590903+0000","last_clean":"2024-09-19T05:40:10.590903+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:40:10.590903+0000","last_undegraded":"2024-09-19T05:40:10.590903+0000","last_fullsized":"2024-09-19T05:40:10.590903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591167+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:40:10.591167+0000","last_peered":"2024-09-19T05:40:10.591167+0000","last_clean":"2024-09-19T05:40:10.591167+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:40:10.591167+0000","last_undegraded":"2024-09-19T05:40:10.591167+0000","last_fullsized":"2024-09-19T05:40:10.591167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594168+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:40:10.594168+0000","last_peered":"2024-09-19T05:40:10.594168+0000","last_clean":"2024-09-19T05:40:10.594168+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:40:10.594168+0000","last_undegraded":"2024-09-19T05:40:10.594168+0000","last_fullsized":"2024-09-19T05:40:10.594168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594696+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:40:10.594696+0000","last_peered":"2024-09-19T05:40:10.594696+0000","last_clean":"2024-09-19T05:40:10.594696+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:40:10.594696+0000","last_undegraded":"2024-09-19T05:40:10.594696+0000","last_fullsized":"2024-09-19T05:40:10.594696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587231+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:40:10.587231+0000","last_peered":"2024-09-19T05:40:10.587231+0000","last_clean":"2024-09-19T05:40:10.587231+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:40:10.587231+0000","last_undegraded":"2024-09-19T05:40:10.587231+0000","last_fullsized":"2024-09-19T05:40:10.587231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591358+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:40:10.591358+0000","last_peered":"2024-09-19T05:40:10.591358+0000","last_clean":"2024-09-19T05:40:10.591358+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:40:10.591358+0000","last_undegraded":"2024-09-19T05:40:10.591358+0000","last_fullsized":"2024-09-19T05:40:10.591358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589448+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:40:10.589448+0000","last_peered":"2024-09-19T05:40:10.589448+0000","last_clean":"2024-09-19T05:40:10.589448+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:40:10.589448+0000","last_undegraded":"2024-09-19T05:40:10.589448+0000","last_fullsized":"2024-09-19T05:40:10.589448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:41.192781+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:40:41.192781+0000","last_peered":"2024-09-19T05:40:41.192781+0000","last_clean":"2024-09-19T05:40:41.192781+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:40:41.192781+0000","last_undegraded":"2024-09-19T05:40:41.192781+0000","last_fullsized":"2024-09-19T05:40:41.192781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64622,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":122,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585946+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:40:10.585946+0000","last_peered":"2024-09-19T05:40:10.585946+0000","last_clean":"2024-09-19T05:40:10.585946+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:40:10.585946+0000","last_undegraded":"2024-09-19T05:40:10.585946+0000","last_fullsized":"2024-09-19T05:40:10.585946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589740+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:40:10.589740+0000","last_peered":"2024-09-19T05:40:10.589740+0000","last_clean":"2024-09-19T05:40:10.589740+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:40:10.589740+0000","last_undegraded":"2024-09-19T05:40:10.589740+0000","last_fullsized":"2024-09-19T05:40:10.589740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592349+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:40:10.592349+0000","last_peered":"2024-09-19T05:40:10.592349+0000","last_clean":"2024-09-19T05:40:10.592349+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:40:10.592349+0000","last_undegraded":"2024-09-19T05:40:10.592349+0000","last_fullsized":"2024-09-19T05:40:10.592349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590509+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:40:10.590509+0000","last_peered":"2024-09-19T05:40:10.590509+0000","last_clean":"2024-09-19T05:40:10.590509+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:40:10.590509+0000","last_undegraded":"2024-09-19T05:40:10.590509+0000","last_fullsized":"2024-09-19T05:40:10.590509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892943+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:40:10.892943+0000","last_peered":"2024-09-19T05:40:10.892943+0000","last_clean":"2024-09-19T05:40:10.892943+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:40:10.892943+0000","last_undegraded":"2024-09-19T05:40:10.892943+0000","last_fullsized":"2024-09-19T05:40:10.892943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590587+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:40:10.590587+0000","last_peered":"2024-09-19T05:40:10.590587+0000","last_clean":"2024-09-19T05:40:10.590587+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:40:10.590587+0000","last_undegraded":"2024-09-19T05:40:10.590587+0000","last_fullsized":"2024-09-19T05:40:10.590587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.893889+0000","last_change":"2024-09-19T05:40:10.893405+0000","last_active":"2024-09-19T05:40:10.893889+0000","last_peered":"2024-09-19T05:40:10.893889+0000","last_clean":"2024-09-19T05:40:10.893889+0000","last_became_active":"2024-09-19T05:40:10.892932+0000","last_became_peered":"2024-09-19T05:40:10.892932+0000","last_unstale":"2024-09-19T05:40:10.893889+0000","last_undegraded":"2024-09-19T05:40:10.893889+0000","last_fullsized":"2024-09-19T05:40:10.893889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:56.153847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591990+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:40:10.591990+0000","last_peered":"2024-09-19T05:40:10.591990+0000","last_clean":"2024-09-19T05:40:10.591990+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:40:10.591990+0000","last_undegraded":"2024-09-19T05:40:10.591990+0000","last_fullsized":"2024-09-19T05:40:10.591990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:41.186745+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:40:41.186745+0000","last_peered":"2024-09-19T05:40:41.186745+0000","last_clean":"2024-09-19T05:40:41.186745+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:40:41.186745+0000","last_undegraded":"2024-09-19T05:40:41.186745+0000","last_fullsized":"2024-09-19T05:40:41.186745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593701+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:40:10.593701+0000","last_peered":"2024-09-19T05:40:10.593701+0000","last_clean":"2024-09-19T05:40:10.593701+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:40:10.593701+0000","last_undegraded":"2024-09-19T05:40:10.593701+0000","last_fullsized":"2024-09-19T05:40:10.593701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892641+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:40:10.892641+0000","last_peered":"2024-09-19T05:40:10.892641+0000","last_clean":"2024-09-19T05:40:10.892641+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:40:10.892641+0000","last_undegraded":"2024-09-19T05:40:10.892641+0000","last_fullsized":"2024-09-19T05:40:10.892641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585692+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:40:10.585692+0000","last_peered":"2024-09-19T05:40:10.585692+0000","last_clean":"2024-09-19T05:40:10.585692+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:40:10.585692+0000","last_undegraded":"2024-09-19T05:40:10.585692+0000","last_fullsized":"2024-09-19T05:40:10.585692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590988+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:40:10.590988+0000","last_peered":"2024-09-19T05:40:10.590988+0000","last_clean":"2024-09-19T05:40:10.590988+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:40:10.590988+0000","last_undegraded":"2024-09-19T05:40:10.590988+0000","last_fullsized":"2024-09-19T05:40:10.590988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594470+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:40:10.594470+0000","last_peered":"2024-09-19T05:40:10.594470+0000","last_clean":"2024-09-19T05:40:10.594470+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:40:10.594470+0000","last_undegraded":"2024-09-19T05:40:10.594470+0000","last_fullsized":"2024-09-19T05:40:10.594470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594185+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:40:10.594185+0000","last_peered":"2024-09-19T05:40:10.594185+0000","last_clean":"2024-09-19T05:40:10.594185+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:40:10.594185+0000","last_undegraded":"2024-09-19T05:40:10.594185+0000","last_fullsized":"2024-09-19T05:40:10.594185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594828+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:40:10.594828+0000","last_peered":"2024-09-19T05:40:10.594828+0000","last_clean":"2024-09-19T05:40:10.594828+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:40:10.594828+0000","last_undegraded":"2024-09-19T05:40:10.594828+0000","last_fullsized":"2024-09-19T05:40:10.594828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587999+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:40:10.587999+0000","last_peered":"2024-09-19T05:40:10.587999+0000","last_clean":"2024-09-19T05:40:10.587999+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:40:10.587999+0000","last_undegraded":"2024-09-19T05:40:10.587999+0000","last_fullsized":"2024-09-19T05:40:10.587999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591660+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:40:10.591660+0000","last_peered":"2024-09-19T05:40:10.591660+0000","last_clean":"2024-09-19T05:40:10.591660+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:40:10.591660+0000","last_undegraded":"2024-09-19T05:40:10.591660+0000","last_fullsized":"2024-09-19T05:40:10.591660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596620+0000","last_change":"2024-09-19T05:40:10.596620+0000","last_active":"2024-09-19T05:40:10.596620+0000","last_peered":"2024-09-19T05:40:10.596620+0000","last_clean":"2024-09-19T05:40:10.596620+0000","last_became_active":"2024-09-19T05:40:10.596126+0000","last_became_peered":"2024-09-19T05:40:10.596126+0000","last_unstale":"2024-09-19T05:40:10.596620+0000","last_undegraded":"2024-09-19T05:40:10.596620+0000","last_fullsized":"2024-09-19T05:40:10.596620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:16.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:36.180279+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:40:36.180279+0000","last_peered":"2024-09-19T05:40:36.180279+0000","last_clean":"2024-09-19T05:40:36.180279+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:40:36.180279+0000","last_undegraded":"2024-09-19T05:40:36.180279+0000","last_fullsized":"2024-09-19T05:40:36.180279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590527+0000","last_change":"2024-09-19T05:40:10.590527+0000","last_active":"2024-09-19T05:40:10.590527+0000","last_peered":"2024-09-19T05:40:10.590527+0000","last_clean":"2024-09-19T05:40:10.590527+0000","last_became_active":"2024-09-19T05:40:10.590181+0000","last_became_peered":"2024-09-19T05:40:10.590181+0000","last_unstale":"2024-09-19T05:40:10.590527+0000","last_undegraded":"2024-09-19T05:40:10.590527+0000","last_fullsized":"2024-09-19T05:40:10.590527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:22:55.157416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587504+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:40:10.587504+0000","last_peered":"2024-09-19T05:40:10.587504+0000","last_clean":"2024-09-19T05:40:10.587504+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:40:10.587504+0000","last_undegraded":"2024-09-19T05:40:10.587504+0000","last_fullsized":"2024-09-19T05:40:10.587504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593931+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:40:10.593931+0000","last_peered":"2024-09-19T05:40:10.593931+0000","last_clean":"2024-09-19T05:40:10.593931+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:40:10.593931+0000","last_undegraded":"2024-09-19T05:40:10.593931+0000","last_fullsized":"2024-09-19T05:40:10.593931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587443+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:40:10.587443+0000","last_peered":"2024-09-19T05:40:10.587443+0000","last_clean":"2024-09-19T05:40:10.587443+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:40:10.587443+0000","last_undegraded":"2024-09-19T05:40:10.587443+0000","last_fullsized":"2024-09-19T05:40:10.587443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.893573+0000","last_change":"2024-09-19T05:40:10.893573+0000","last_active":"2024-09-19T05:40:10.893573+0000","last_peered":"2024-09-19T05:40:10.893573+0000","last_clean":"2024-09-19T05:40:10.893573+0000","last_became_active":"2024-09-19T05:40:10.893216+0000","last_became_peered":"2024-09-19T05:40:10.893216+0000","last_unstale":"2024-09-19T05:40:10.893573+0000","last_undegraded":"2024-09-19T05:40:10.893573+0000","last_fullsized":"2024-09-19T05:40:10.893573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:50.147170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892318+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:40:10.892318+0000","last_peered":"2024-09-19T05:40:10.892318+0000","last_clean":"2024-09-19T05:40:10.892318+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:40:10.892318+0000","last_undegraded":"2024-09-19T05:40:10.892318+0000","last_fullsized":"2024-09-19T05:40:10.892318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595266+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:40:10.595266+0000","last_peered":"2024-09-19T05:40:10.595266+0000","last_clean":"2024-09-19T05:40:10.595266+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:40:10.595266+0000","last_undegraded":"2024-09-19T05:40:10.595266+0000","last_fullsized":"2024-09-19T05:40:10.595266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594480+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:40:10.594480+0000","last_peered":"2024-09-19T05:40:10.594480+0000","last_clean":"2024-09-19T05:40:10.594480+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:40:10.594480+0000","last_undegraded":"2024-09-19T05:40:10.594480+0000","last_fullsized":"2024-09-19T05:40:10.594480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588192+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:40:10.588192+0000","last_peered":"2024-09-19T05:40:10.588192+0000","last_clean":"2024-09-19T05:40:10.588192+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:40:10.588192+0000","last_undegraded":"2024-09-19T05:40:10.588192+0000","last_fullsized":"2024-09-19T05:40:10.588192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594240+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:40:10.594240+0000","last_peered":"2024-09-19T05:40:10.594240+0000","last_clean":"2024-09-19T05:40:10.594240+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:40:10.594240+0000","last_undegraded":"2024-09-19T05:40:10.594240+0000","last_fullsized":"2024-09-19T05:40:10.594240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590867+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:40:10.590867+0000","last_peered":"2024-09-19T05:40:10.590867+0000","last_clean":"2024-09-19T05:40:10.590867+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:40:10.590867+0000","last_undegraded":"2024-09-19T05:40:10.590867+0000","last_fullsized":"2024-09-19T05:40:10.590867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594398+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:40:10.594398+0000","last_peered":"2024-09-19T05:40:10.594398+0000","last_clean":"2024-09-19T05:40:10.594398+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:40:10.594398+0000","last_undegraded":"2024-09-19T05:40:10.594398+0000","last_fullsized":"2024-09-19T05:40:10.594398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591310+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:40:10.591310+0000","last_peered":"2024-09-19T05:40:10.591310+0000","last_clean":"2024-09-19T05:40:10.591310+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:40:10.591310+0000","last_undegraded":"2024-09-19T05:40:10.591310+0000","last_fullsized":"2024-09-19T05:40:10.591310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595203+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:40:10.595203+0000","last_peered":"2024-09-19T05:40:10.595203+0000","last_clean":"2024-09-19T05:40:10.595203+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:40:10.595203+0000","last_undegraded":"2024-09-19T05:40:10.595203+0000","last_fullsized":"2024-09-19T05:40:10.595203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594790+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:40:10.594790+0000","last_peered":"2024-09-19T05:40:10.594790+0000","last_clean":"2024-09-19T05:40:10.594790+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:40:10.594790+0000","last_undegraded":"2024-09-19T05:40:10.594790+0000","last_fullsized":"2024-09-19T05:40:10.594790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587984+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:40:10.587984+0000","last_peered":"2024-09-19T05:40:10.587984+0000","last_clean":"2024-09-19T05:40:10.587984+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:40:10.587984+0000","last_undegraded":"2024-09-19T05:40:10.587984+0000","last_fullsized":"2024-09-19T05:40:10.587984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586387+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:40:10.586387+0000","last_peered":"2024-09-19T05:40:10.586387+0000","last_clean":"2024-09-19T05:40:10.586387+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:40:10.586387+0000","last_undegraded":"2024-09-19T05:40:10.586387+0000","last_fullsized":"2024-09-19T05:40:10.586387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594933+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:40:10.594933+0000","last_peered":"2024-09-19T05:40:10.594933+0000","last_clean":"2024-09-19T05:40:10.594933+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:40:10.594933+0000","last_undegraded":"2024-09-19T05:40:10.594933+0000","last_fullsized":"2024-09-19T05:40:10.594933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589559+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:40:10.589559+0000","last_peered":"2024-09-19T05:40:10.589559+0000","last_clean":"2024-09-19T05:40:10.589559+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:40:10.589559+0000","last_undegraded":"2024-09-19T05:40:10.589559+0000","last_fullsized":"2024-09-19T05:40:10.589559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.891939+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:40:10.891939+0000","last_peered":"2024-09-19T05:40:10.891939+0000","last_clean":"2024-09-19T05:40:10.891939+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:40:10.891939+0000","last_undegraded":"2024-09-19T05:40:10.891939+0000","last_fullsized":"2024-09-19T05:40:10.891939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892024+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:40:10.892024+0000","last_peered":"2024-09-19T05:40:10.892024+0000","last_clean":"2024-09-19T05:40:10.892024+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:40:10.892024+0000","last_undegraded":"2024-09-19T05:40:10.892024+0000","last_fullsized":"2024-09-19T05:40:10.892024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595181+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:40:10.595181+0000","last_peered":"2024-09-19T05:40:10.595181+0000","last_clean":"2024-09-19T05:40:10.595181+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:40:10.595181+0000","last_undegraded":"2024-09-19T05:40:10.595181+0000","last_fullsized":"2024-09-19T05:40:10.595181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591269+0000","last_change":"2024-09-19T05:40:10.590802+0000","last_active":"2024-09-19T05:40:10.591269+0000","last_peered":"2024-09-19T05:40:10.591269+0000","last_clean":"2024-09-19T05:40:10.591269+0000","last_became_active":"2024-09-19T05:40:10.590515+0000","last_became_peered":"2024-09-19T05:40:10.590515+0000","last_unstale":"2024-09-19T05:40:10.591269+0000","last_undegraded":"2024-09-19T05:40:10.591269+0000","last_fullsized":"2024-09-19T05:40:10.591269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:37.010052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591075+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:40:10.591075+0000","last_peered":"2024-09-19T05:40:10.591075+0000","last_clean":"2024-09-19T05:40:10.591075+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:40:10.591075+0000","last_undegraded":"2024-09-19T05:40:10.591075+0000","last_fullsized":"2024-09-19T05:40:10.591075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589178+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:40:10.589178+0000","last_peered":"2024-09-19T05:40:10.589178+0000","last_clean":"2024-09-19T05:40:10.589178+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:40:10.589178+0000","last_undegraded":"2024-09-19T05:40:10.589178+0000","last_fullsized":"2024-09-19T05:40:10.589178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592427+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:40:10.592427+0000","last_peered":"2024-09-19T05:40:10.592427+0000","last_clean":"2024-09-19T05:40:10.592427+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:40:10.592427+0000","last_undegraded":"2024-09-19T05:40:10.592427+0000","last_fullsized":"2024-09-19T05:40:10.592427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593998+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:40:10.593998+0000","last_peered":"2024-09-19T05:40:10.593998+0000","last_clean":"2024-09-19T05:40:10.593998+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:40:10.593998+0000","last_undegraded":"2024-09-19T05:40:10.593998+0000","last_fullsized":"2024-09-19T05:40:10.593998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587111+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:40:10.587111+0000","last_peered":"2024-09-19T05:40:10.587111+0000","last_clean":"2024-09-19T05:40:10.587111+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:40:10.587111+0000","last_undegraded":"2024-09-19T05:40:10.587111+0000","last_fullsized":"2024-09-19T05:40:10.587111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587886+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:40:10.587886+0000","last_peered":"2024-09-19T05:40:10.587886+0000","last_clean":"2024-09-19T05:40:10.587886+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:40:10.587886+0000","last_undegraded":"2024-09-19T05:40:10.587886+0000","last_fullsized":"2024-09-19T05:40:10.587886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589254+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:40:10.589254+0000","last_peered":"2024-09-19T05:40:10.589254+0000","last_clean":"2024-09-19T05:40:10.589254+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:40:10.589254+0000","last_undegraded":"2024-09-19T05:40:10.589254+0000","last_fullsized":"2024-09-19T05:40:10.589254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588920+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:40:10.588920+0000","last_peered":"2024-09-19T05:40:10.588920+0000","last_clean":"2024-09-19T05:40:10.588920+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:40:10.588920+0000","last_undegraded":"2024-09-19T05:40:10.588920+0000","last_fullsized":"2024-09-19T05:40:10.588920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892539+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:40:10.892539+0000","last_peered":"2024-09-19T05:40:10.892539+0000","last_clean":"2024-09-19T05:40:10.892539+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:40:10.892539+0000","last_undegraded":"2024-09-19T05:40:10.892539+0000","last_fullsized":"2024-09-19T05:40:10.892539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892708+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:40:10.892708+0000","last_peered":"2024-09-19T05:40:10.892708+0000","last_clean":"2024-09-19T05:40:10.892708+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:40:10.892708+0000","last_undegraded":"2024-09-19T05:40:10.892708+0000","last_fullsized":"2024-09-19T05:40:10.892708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594729+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:40:10.594729+0000","last_peered":"2024-09-19T05:40:10.594729+0000","last_clean":"2024-09-19T05:40:10.594729+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:40:10.594729+0000","last_undegraded":"2024-09-19T05:40:10.594729+0000","last_fullsized":"2024-09-19T05:40:10.594729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72616,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":570,"num_read_kb":370,"num_write":252,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547231,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":499829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1151267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939692,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":503462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":499744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1634304,"data_stored":1097029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":495263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1091825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":499337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":494578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1105920,"data_stored":561528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1085440,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":496719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:40:42.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:40:42.962 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:43.084 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:42 smithi038 ceph-mon[31227]: pgmap v335: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:43.084 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:42 smithi038 ceph-mon[31227]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:43.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:42 smithi179 ceph-mon[35781]: pgmap v335: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:43.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:42 smithi179 ceph-mon[35781]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:43.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:42 smithi047 ceph-mon[35327]: pgmap v335: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:42 smithi047 ceph-mon[35327]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- 172.21.15.38:0/1877139178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 msgr2=0x7fdadc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 --2- 172.21.15.38:0/1877139178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc1067e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009900 tx=0x7fdad002a410 comp rx=0 tx=0).stop 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- 172.21.15.38:0/1877139178 shutdown_connections 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 --2- 172.21.15.38:0/1877139178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc106f10 0x7fdadc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 --2- 172.21.15.38:0/1877139178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc1067e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 --2- 172.21.15.38:0/1877139178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc101de0 0x7fdadc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- 172.21.15.38:0/1877139178 >> 172.21.15.38:0/1877139178 conn(0x7fdadc0fd8c0 msgr2=0x7fdadc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:43.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- 172.21.15.38:0/1877139178 shutdown_connections 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- 172.21.15.38:0/1877139178 wait complete. 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 Processor -- start 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.439+0000 7fdae2280640 1 -- start start 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc101de0 0x7fdadc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc106f10 0x7fdadc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc111f70 con 0x7fdadc102790 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc111df0 con 0x7fdadc101de0 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae2280640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1120f0 con 0x7fdadc106f10 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42872/0 (socket says 172.21.15.38:42872) 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 -- 172.21.15.38:0/1378694179 learned_addr learned my addr 172.21.15.38:0/1378694179 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:43.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadbfff640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc101de0 0x7fdadc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdae081d640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc106f10 0x7fdadc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 -- 172.21.15.38:0/1378694179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc106f10 msgr2=0x7fdadc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc106f10 0x7fdadc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 -- 172.21.15.38:0/1378694179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc101de0 msgr2=0x7fdadc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc101de0 0x7fdadc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 -- 172.21.15.38:0/1378694179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc1a6950 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.440+0000 7fdadb7fe640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006010 tx=0x7fdad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.441+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad003f030 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.441+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0003ce0 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.441+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc1a6be0 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.441+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0035070 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.441+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc1a70c0 con 0x7fdadc102790 2024-09-19T05:40:43.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.442+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc1021c0 con 0x7fdadc102790 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.443+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdad0042030 con 0x7fdadc102790 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.444+0000 7fdad97fa640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 0x7fdac40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.444+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdad00f0510 con 0x7fdadc102790 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.444+0000 7fdadbfff640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 0x7fdac40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.444+0000 7fdadbfff640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 0x7fdac40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009bd0 tx=0x7fdacc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.447+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdad00bc7e0 con 0x7fdadc102790 2024-09-19T05:40:43.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.619+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdadc062630 con 0x7fdac40a32d0 2024-09-19T05:40:43.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.627+0000 7fdad97fa640 1 -- 172.21.15.38:0/1378694179 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7fdadc062630 con 0x7fdac40a32d0 2024-09-19T05:40:43.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:43.634 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:40:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 msgr2=0x7fdac40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 0x7fdac40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009bd0 tx=0x7fdacc005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 msgr2=0x7fdadc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006010 tx=0x7fdad00042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 shutdown_connections 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdac40a32d0 0x7fdac40a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdadc106f10 0x7fdadc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdadc102790 0x7fdadc074a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 --2- 172.21.15.38:0/1378694179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdadc101de0 0x7fdadc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 >> 172.21.15.38:0/1378694179 conn(0x7fdadc0fd8c0 msgr2=0x7fdadc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 shutdown_connections 2024-09-19T05:40:43.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:43.631+0000 7fdae2280640 1 -- 172.21.15.38:0/1378694179 wait complete. 2024-09-19T05:40:44.339 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-19T05:40:43.609174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663218,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":711,"num_read_kb":479,"num_write":435,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":356,"ondisk_log_size":356,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355056,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586768,"statfs":{"total":1151940427776,"available":1151576850432,"internally_reserved":0,"allocated":14385152,"data_stored":7951496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003497"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:40.184549+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:40:40.184549+0000","last_peered":"2024-09-19T05:40:40.184549+0000","last_clean":"2024-09-19T05:40:40.184549+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:40:40.184549+0000","last_undegraded":"2024-09-19T05:40:40.184549+0000","last_fullsized":"2024-09-19T05:40:40.184549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593599+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:40:10.593599+0000","last_peered":"2024-09-19T05:40:10.593599+0000","last_clean":"2024-09-19T05:40:10.593599+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:40:10.593599+0000","last_undegraded":"2024-09-19T05:40:10.593599+0000","last_fullsized":"2024-09-19T05:40:10.593599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587091+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:40:10.587091+0000","last_peered":"2024-09-19T05:40:10.587091+0000","last_clean":"2024-09-19T05:40:10.587091+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:40:10.587091+0000","last_undegraded":"2024-09-19T05:40:10.587091+0000","last_fullsized":"2024-09-19T05:40:10.587091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593833+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:40:10.593833+0000","last_peered":"2024-09-19T05:40:10.593833+0000","last_clean":"2024-09-19T05:40:10.593833+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:40:10.593833+0000","last_undegraded":"2024-09-19T05:40:10.593833+0000","last_fullsized":"2024-09-19T05:40:10.593833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892882+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:40:10.892882+0000","last_peered":"2024-09-19T05:40:10.892882+0000","last_clean":"2024-09-19T05:40:10.892882+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:40:10.892882+0000","last_undegraded":"2024-09-19T05:40:10.892882+0000","last_fullsized":"2024-09-19T05:40:10.892882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588014+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:40:10.588014+0000","last_peered":"2024-09-19T05:40:10.588014+0000","last_clean":"2024-09-19T05:40:10.588014+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:40:10.588014+0000","last_undegraded":"2024-09-19T05:40:10.588014+0000","last_fullsized":"2024-09-19T05:40:10.588014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589582+0000","last_change":"2024-09-19T05:38:59.855714+0000","last_active":"2024-09-19T05:40:10.589582+0000","last_peered":"2024-09-19T05:40:10.589582+0000","last_clean":"2024-09-19T05:40:10.589582+0000","last_became_active":"2024-09-19T05:38:59.855268+0000","last_became_peered":"2024-09-19T05:38:59.855268+0000","last_unstale":"2024-09-19T05:40:10.589582+0000","last_undegraded":"2024-09-19T05:40:10.589582+0000","last_fullsized":"2024-09-19T05:40:10.589582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:20.050068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589637+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:40:10.589637+0000","last_peered":"2024-09-19T05:40:10.589637+0000","last_clean":"2024-09-19T05:40:10.589637+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:40:10.589637+0000","last_undegraded":"2024-09-19T05:40:10.589637+0000","last_fullsized":"2024-09-19T05:40:10.589637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595400+0000","last_change":"2024-09-19T05:40:10.595400+0000","last_active":"2024-09-19T05:40:10.595400+0000","last_peered":"2024-09-19T05:40:10.595400+0000","last_clean":"2024-09-19T05:40:10.595400+0000","last_became_active":"2024-09-19T05:40:10.595015+0000","last_became_peered":"2024-09-19T05:40:10.595015+0000","last_unstale":"2024-09-19T05:40:10.595400+0000","last_undegraded":"2024-09-19T05:40:10.595400+0000","last_fullsized":"2024-09-19T05:40:10.595400+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.449136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588735+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:40:10.588735+0000","last_peered":"2024-09-19T05:40:10.588735+0000","last_clean":"2024-09-19T05:40:10.588735+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:40:10.588735+0000","last_undegraded":"2024-09-19T05:40:10.588735+0000","last_fullsized":"2024-09-19T05:40:10.588735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591458+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:40:10.591458+0000","last_peered":"2024-09-19T05:40:10.591458+0000","last_clean":"2024-09-19T05:40:10.591458+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:40:10.591458+0000","last_undegraded":"2024-09-19T05:40:10.591458+0000","last_fullsized":"2024-09-19T05:40:10.591458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588329+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:40:10.588329+0000","last_peered":"2024-09-19T05:40:10.588329+0000","last_clean":"2024-09-19T05:40:10.588329+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:40:10.588329+0000","last_undegraded":"2024-09-19T05:40:10.588329+0000","last_fullsized":"2024-09-19T05:40:10.588329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587299+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:40:10.587299+0000","last_peered":"2024-09-19T05:40:10.587299+0000","last_clean":"2024-09-19T05:40:10.587299+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:40:10.587299+0000","last_undegraded":"2024-09-19T05:40:10.587299+0000","last_fullsized":"2024-09-19T05:40:10.587299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589368+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:40:10.589368+0000","last_peered":"2024-09-19T05:40:10.589368+0000","last_clean":"2024-09-19T05:40:10.589368+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:40:10.589368+0000","last_undegraded":"2024-09-19T05:40:10.589368+0000","last_fullsized":"2024-09-19T05:40:10.589368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588655+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:40:10.588655+0000","last_peered":"2024-09-19T05:40:10.588655+0000","last_clean":"2024-09-19T05:40:10.588655+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:40:10.588655+0000","last_undegraded":"2024-09-19T05:40:10.588655+0000","last_fullsized":"2024-09-19T05:40:10.588655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593712+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:40:10.593712+0000","last_peered":"2024-09-19T05:40:10.593712+0000","last_clean":"2024-09-19T05:40:10.593712+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:40:10.593712+0000","last_undegraded":"2024-09-19T05:40:10.593712+0000","last_fullsized":"2024-09-19T05:40:10.593712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588120+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:40:10.588120+0000","last_peered":"2024-09-19T05:40:10.588120+0000","last_clean":"2024-09-19T05:40:10.588120+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:40:10.588120+0000","last_undegraded":"2024-09-19T05:40:10.588120+0000","last_fullsized":"2024-09-19T05:40:10.588120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588698+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:40:10.588698+0000","last_peered":"2024-09-19T05:40:10.588698+0000","last_clean":"2024-09-19T05:40:10.588698+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:40:10.588698+0000","last_undegraded":"2024-09-19T05:40:10.588698+0000","last_fullsized":"2024-09-19T05:40:10.588698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594126+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:40:10.594126+0000","last_peered":"2024-09-19T05:40:10.594126+0000","last_clean":"2024-09-19T05:40:10.594126+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:40:10.594126+0000","last_undegraded":"2024-09-19T05:40:10.594126+0000","last_fullsized":"2024-09-19T05:40:10.594126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594555+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:40:10.594555+0000","last_peered":"2024-09-19T05:40:10.594555+0000","last_clean":"2024-09-19T05:40:10.594555+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:40:10.594555+0000","last_undegraded":"2024-09-19T05:40:10.594555+0000","last_fullsized":"2024-09-19T05:40:10.594555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585984+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:40:10.585984+0000","last_peered":"2024-09-19T05:40:10.585984+0000","last_clean":"2024-09-19T05:40:10.585984+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:40:10.585984+0000","last_undegraded":"2024-09-19T05:40:10.585984+0000","last_fullsized":"2024-09-19T05:40:10.585984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587945+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:40:10.587945+0000","last_peered":"2024-09-19T05:40:10.587945+0000","last_clean":"2024-09-19T05:40:10.587945+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:40:10.587945+0000","last_undegraded":"2024-09-19T05:40:10.587945+0000","last_fullsized":"2024-09-19T05:40:10.587945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589924+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:40:10.589924+0000","last_peered":"2024-09-19T05:40:10.589924+0000","last_clean":"2024-09-19T05:40:10.589924+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:40:10.589924+0000","last_undegraded":"2024-09-19T05:40:10.589924+0000","last_fullsized":"2024-09-19T05:40:10.589924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588877+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:40:10.588877+0000","last_peered":"2024-09-19T05:40:10.588877+0000","last_clean":"2024-09-19T05:40:10.588877+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:40:10.588877+0000","last_undegraded":"2024-09-19T05:40:10.588877+0000","last_fullsized":"2024-09-19T05:40:10.588877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892768+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:40:10.892768+0000","last_peered":"2024-09-19T05:40:10.892768+0000","last_clean":"2024-09-19T05:40:10.892768+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:40:10.892768+0000","last_undegraded":"2024-09-19T05:40:10.892768+0000","last_fullsized":"2024-09-19T05:40:10.892768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589464+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:40:10.589464+0000","last_peered":"2024-09-19T05:40:10.589464+0000","last_clean":"2024-09-19T05:40:10.589464+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:40:10.589464+0000","last_undegraded":"2024-09-19T05:40:10.589464+0000","last_fullsized":"2024-09-19T05:40:10.589464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587217+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:40:10.587217+0000","last_peered":"2024-09-19T05:40:10.587217+0000","last_clean":"2024-09-19T05:40:10.587217+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:40:10.587217+0000","last_undegraded":"2024-09-19T05:40:10.587217+0000","last_fullsized":"2024-09-19T05:40:10.587217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594349+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:40:10.594349+0000","last_peered":"2024-09-19T05:40:10.594349+0000","last_clean":"2024-09-19T05:40:10.594349+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:40:10.594349+0000","last_undegraded":"2024-09-19T05:40:10.594349+0000","last_fullsized":"2024-09-19T05:40:10.594349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586477+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:40:10.586477+0000","last_peered":"2024-09-19T05:40:10.586477+0000","last_clean":"2024-09-19T05:40:10.586477+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:40:10.586477+0000","last_undegraded":"2024-09-19T05:40:10.586477+0000","last_fullsized":"2024-09-19T05:40:10.586477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593449+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:40:10.593449+0000","last_peered":"2024-09-19T05:40:10.593449+0000","last_clean":"2024-09-19T05:40:10.593449+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:40:10.593449+0000","last_undegraded":"2024-09-19T05:40:10.593449+0000","last_fullsized":"2024-09-19T05:40:10.593449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595364+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:40:10.595364+0000","last_peered":"2024-09-19T05:40:10.595364+0000","last_clean":"2024-09-19T05:40:10.595364+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:40:10.595364+0000","last_undegraded":"2024-09-19T05:40:10.595364+0000","last_fullsized":"2024-09-19T05:40:10.595364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589625+0000","last_change":"2024-09-19T05:39:00.095972+0000","last_active":"2024-09-19T05:40:10.589625+0000","last_peered":"2024-09-19T05:40:10.589625+0000","last_clean":"2024-09-19T05:40:10.589625+0000","last_became_active":"2024-09-19T05:39:00.095558+0000","last_became_peered":"2024-09-19T05:39:00.095558+0000","last_unstale":"2024-09-19T05:40:10.589625+0000","last_undegraded":"2024-09-19T05:40:10.589625+0000","last_fullsized":"2024-09-19T05:40:10.589625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:13.708685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594520+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:40:10.594520+0000","last_peered":"2024-09-19T05:40:10.594520+0000","last_clean":"2024-09-19T05:40:10.594520+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:40:10.594520+0000","last_undegraded":"2024-09-19T05:40:10.594520+0000","last_fullsized":"2024-09-19T05:40:10.594520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588685+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:40:10.588685+0000","last_peered":"2024-09-19T05:40:10.588685+0000","last_clean":"2024-09-19T05:40:10.588685+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:40:10.588685+0000","last_undegraded":"2024-09-19T05:40:10.588685+0000","last_fullsized":"2024-09-19T05:40:10.588685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586324+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:40:10.586324+0000","last_peered":"2024-09-19T05:40:10.586324+0000","last_clean":"2024-09-19T05:40:10.586324+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:40:10.586324+0000","last_undegraded":"2024-09-19T05:40:10.586324+0000","last_fullsized":"2024-09-19T05:40:10.586324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587712+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:40:10.587712+0000","last_peered":"2024-09-19T05:40:10.587712+0000","last_clean":"2024-09-19T05:40:10.587712+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:40:10.587712+0000","last_undegraded":"2024-09-19T05:40:10.587712+0000","last_fullsized":"2024-09-19T05:40:10.587712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588398+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:40:10.588398+0000","last_peered":"2024-09-19T05:40:10.588398+0000","last_clean":"2024-09-19T05:40:10.588398+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:40:10.588398+0000","last_undegraded":"2024-09-19T05:40:10.588398+0000","last_fullsized":"2024-09-19T05:40:10.588398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:38.182517+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:40:38.182517+0000","last_peered":"2024-09-19T05:40:38.182517+0000","last_clean":"2024-09-19T05:40:38.182517+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:40:38.182517+0000","last_undegraded":"2024-09-19T05:40:38.182517+0000","last_fullsized":"2024-09-19T05:40:38.182517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595102+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:40:10.595102+0000","last_peered":"2024-09-19T05:40:10.595102+0000","last_clean":"2024-09-19T05:40:10.595102+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:40:10.595102+0000","last_undegraded":"2024-09-19T05:40:10.595102+0000","last_fullsized":"2024-09-19T05:40:10.595102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594007+0000","last_change":"2024-09-19T05:40:10.594007+0000","last_active":"2024-09-19T05:40:10.594007+0000","last_peered":"2024-09-19T05:40:10.594007+0000","last_clean":"2024-09-19T05:40:10.594007+0000","last_became_active":"2024-09-19T05:40:10.593493+0000","last_became_peered":"2024-09-19T05:40:10.593493+0000","last_unstale":"2024-09-19T05:40:10.594007+0000","last_undegraded":"2024-09-19T05:40:10.594007+0000","last_fullsized":"2024-09-19T05:40:10.594007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:58.225840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:38.181872+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:40:38.181872+0000","last_peered":"2024-09-19T05:40:38.181872+0000","last_clean":"2024-09-19T05:40:38.181872+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:40:38.181872+0000","last_undegraded":"2024-09-19T05:40:38.181872+0000","last_fullsized":"2024-09-19T05:40:38.181872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594940+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:40:10.594940+0000","last_peered":"2024-09-19T05:40:10.594940+0000","last_clean":"2024-09-19T05:40:10.594940+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:40:10.594940+0000","last_undegraded":"2024-09-19T05:40:10.594940+0000","last_fullsized":"2024-09-19T05:40:10.594940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594704+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:40:10.594704+0000","last_peered":"2024-09-19T05:40:10.594704+0000","last_clean":"2024-09-19T05:40:10.594704+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:40:10.594704+0000","last_undegraded":"2024-09-19T05:40:10.594704+0000","last_fullsized":"2024-09-19T05:40:10.594704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587062+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:40:10.587062+0000","last_peered":"2024-09-19T05:40:10.587062+0000","last_clean":"2024-09-19T05:40:10.587062+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:40:10.587062+0000","last_undegraded":"2024-09-19T05:40:10.587062+0000","last_fullsized":"2024-09-19T05:40:10.587062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592345+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:40:10.592345+0000","last_peered":"2024-09-19T05:40:10.592345+0000","last_clean":"2024-09-19T05:40:10.592345+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:40:10.592345+0000","last_undegraded":"2024-09-19T05:40:10.592345+0000","last_fullsized":"2024-09-19T05:40:10.592345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593560+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:40:10.593560+0000","last_peered":"2024-09-19T05:40:10.593560+0000","last_clean":"2024-09-19T05:40:10.593560+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:40:10.593560+0000","last_undegraded":"2024-09-19T05:40:10.593560+0000","last_fullsized":"2024-09-19T05:40:10.593560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590772+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:40:10.590772+0000","last_peered":"2024-09-19T05:40:10.590772+0000","last_clean":"2024-09-19T05:40:10.590772+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:40:10.590772+0000","last_undegraded":"2024-09-19T05:40:10.590772+0000","last_fullsized":"2024-09-19T05:40:10.590772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587267+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:40:10.587267+0000","last_peered":"2024-09-19T05:40:10.587267+0000","last_clean":"2024-09-19T05:40:10.587267+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:40:10.587267+0000","last_undegraded":"2024-09-19T05:40:10.587267+0000","last_fullsized":"2024-09-19T05:40:10.587267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590512+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:40:10.590512+0000","last_peered":"2024-09-19T05:40:10.590512+0000","last_clean":"2024-09-19T05:40:10.590512+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:40:10.590512+0000","last_undegraded":"2024-09-19T05:40:10.590512+0000","last_fullsized":"2024-09-19T05:40:10.590512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591099+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:40:10.591099+0000","last_peered":"2024-09-19T05:40:10.591099+0000","last_clean":"2024-09-19T05:40:10.591099+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:40:10.591099+0000","last_undegraded":"2024-09-19T05:40:10.591099+0000","last_fullsized":"2024-09-19T05:40:10.591099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:42.207659+0000","last_change":"2024-09-19T05:39:00.097011+0000","last_active":"2024-09-19T05:40:42.207659+0000","last_peered":"2024-09-19T05:40:42.207659+0000","last_clean":"2024-09-19T05:40:42.207659+0000","last_became_active":"2024-09-19T05:39:00.096349+0000","last_became_peered":"2024-09-19T05:39:00.096349+0000","last_unstale":"2024-09-19T05:40:42.207659+0000","last_undegraded":"2024-09-19T05:40:42.207659+0000","last_fullsized":"2024-09-19T05:40:42.207659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:45.484368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596589+0000","last_change":"2024-09-19T05:40:10.596589+0000","last_active":"2024-09-19T05:40:10.596589+0000","last_peered":"2024-09-19T05:40:10.596589+0000","last_clean":"2024-09-19T05:40:10.596589+0000","last_became_active":"2024-09-19T05:40:10.596087+0000","last_became_peered":"2024-09-19T05:40:10.596087+0000","last_unstale":"2024-09-19T05:40:10.596589+0000","last_undegraded":"2024-09-19T05:40:10.596589+0000","last_fullsized":"2024-09-19T05:40:10.596589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594372+0000","last_change":"2024-09-19T05:40:10.594372+0000","last_active":"2024-09-19T05:40:10.594372+0000","last_peered":"2024-09-19T05:40:10.594372+0000","last_clean":"2024-09-19T05:40:10.594372+0000","last_became_active":"2024-09-19T05:40:10.594018+0000","last_became_peered":"2024-09-19T05:40:10.594018+0000","last_unstale":"2024-09-19T05:40:10.594372+0000","last_undegraded":"2024-09-19T05:40:10.594372+0000","last_fullsized":"2024-09-19T05:40:10.594372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:23.706247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592489+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:40:10.592489+0000","last_peered":"2024-09-19T05:40:10.592489+0000","last_clean":"2024-09-19T05:40:10.592489+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:40:10.592489+0000","last_undegraded":"2024-09-19T05:40:10.592489+0000","last_fullsized":"2024-09-19T05:40:10.592489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892210+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:40:10.892210+0000","last_peered":"2024-09-19T05:40:10.892210+0000","last_clean":"2024-09-19T05:40:10.892210+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:40:10.892210+0000","last_undegraded":"2024-09-19T05:40:10.892210+0000","last_fullsized":"2024-09-19T05:40:10.892210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590878+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:40:10.590878+0000","last_peered":"2024-09-19T05:40:10.590878+0000","last_clean":"2024-09-19T05:40:10.590878+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:40:10.590878+0000","last_undegraded":"2024-09-19T05:40:10.590878+0000","last_fullsized":"2024-09-19T05:40:10.590878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595364+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:40:10.595364+0000","last_peered":"2024-09-19T05:40:10.595364+0000","last_clean":"2024-09-19T05:40:10.595364+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:40:10.595364+0000","last_undegraded":"2024-09-19T05:40:10.595364+0000","last_fullsized":"2024-09-19T05:40:10.595364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590770+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:40:10.590770+0000","last_peered":"2024-09-19T05:40:10.590770+0000","last_clean":"2024-09-19T05:40:10.590770+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:40:10.590770+0000","last_undegraded":"2024-09-19T05:40:10.590770+0000","last_fullsized":"2024-09-19T05:40:10.590770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594949+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:40:10.594949+0000","last_peered":"2024-09-19T05:40:10.594949+0000","last_clean":"2024-09-19T05:40:10.594949+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:40:10.594949+0000","last_undegraded":"2024-09-19T05:40:10.594949+0000","last_fullsized":"2024-09-19T05:40:10.594949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589482+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:40:10.589482+0000","last_peered":"2024-09-19T05:40:10.589482+0000","last_clean":"2024-09-19T05:40:10.589482+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:40:10.589482+0000","last_undegraded":"2024-09-19T05:40:10.589482+0000","last_fullsized":"2024-09-19T05:40:10.589482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590244+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:40:10.590244+0000","last_peered":"2024-09-19T05:40:10.590244+0000","last_clean":"2024-09-19T05:40:10.590244+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:40:10.590244+0000","last_undegraded":"2024-09-19T05:40:10.590244+0000","last_fullsized":"2024-09-19T05:40:10.590244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596504+0000","last_change":"2024-09-19T05:40:10.596504+0000","last_active":"2024-09-19T05:40:10.596504+0000","last_peered":"2024-09-19T05:40:10.596504+0000","last_clean":"2024-09-19T05:40:10.596504+0000","last_became_active":"2024-09-19T05:40:10.596112+0000","last_became_peered":"2024-09-19T05:40:10.596112+0000","last_unstale":"2024-09-19T05:40:10.596504+0000","last_undegraded":"2024-09-19T05:40:10.596504+0000","last_fullsized":"2024-09-19T05:40:10.596504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:02.818567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587791+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:40:10.587791+0000","last_peered":"2024-09-19T05:40:10.587791+0000","last_clean":"2024-09-19T05:40:10.587791+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:40:10.587791+0000","last_undegraded":"2024-09-19T05:40:10.587791+0000","last_fullsized":"2024-09-19T05:40:10.587791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595308+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:40:10.595308+0000","last_peered":"2024-09-19T05:40:10.595308+0000","last_clean":"2024-09-19T05:40:10.595308+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:40:10.595308+0000","last_undegraded":"2024-09-19T05:40:10.595308+0000","last_fullsized":"2024-09-19T05:40:10.595308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586964+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:40:10.586964+0000","last_peered":"2024-09-19T05:40:10.586964+0000","last_clean":"2024-09-19T05:40:10.586964+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:40:10.586964+0000","last_undegraded":"2024-09-19T05:40:10.586964+0000","last_fullsized":"2024-09-19T05:40:10.586964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":99,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:42.187615+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:40:42.187615+0000","last_peered":"2024-09-19T05:40:42.187615+0000","last_clean":"2024-09-19T05:40:42.187615+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:40:42.187615+0000","last_undegraded":"2024-09-19T05:40:42.187615+0000","last_fullsized":"2024-09-19T05:40:42.187615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586098+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:40:10.586098+0000","last_peered":"2024-09-19T05:40:10.586098+0000","last_clean":"2024-09-19T05:40:10.586098+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:40:10.586098+0000","last_undegraded":"2024-09-19T05:40:10.586098+0000","last_fullsized":"2024-09-19T05:40:10.586098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590903+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:40:10.590903+0000","last_peered":"2024-09-19T05:40:10.590903+0000","last_clean":"2024-09-19T05:40:10.590903+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:40:10.590903+0000","last_undegraded":"2024-09-19T05:40:10.590903+0000","last_fullsized":"2024-09-19T05:40:10.590903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591167+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:40:10.591167+0000","last_peered":"2024-09-19T05:40:10.591167+0000","last_clean":"2024-09-19T05:40:10.591167+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:40:10.591167+0000","last_undegraded":"2024-09-19T05:40:10.591167+0000","last_fullsized":"2024-09-19T05:40:10.591167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594168+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:40:10.594168+0000","last_peered":"2024-09-19T05:40:10.594168+0000","last_clean":"2024-09-19T05:40:10.594168+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:40:10.594168+0000","last_undegraded":"2024-09-19T05:40:10.594168+0000","last_fullsized":"2024-09-19T05:40:10.594168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594696+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:40:10.594696+0000","last_peered":"2024-09-19T05:40:10.594696+0000","last_clean":"2024-09-19T05:40:10.594696+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:40:10.594696+0000","last_undegraded":"2024-09-19T05:40:10.594696+0000","last_fullsized":"2024-09-19T05:40:10.594696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587231+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:40:10.587231+0000","last_peered":"2024-09-19T05:40:10.587231+0000","last_clean":"2024-09-19T05:40:10.587231+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:40:10.587231+0000","last_undegraded":"2024-09-19T05:40:10.587231+0000","last_fullsized":"2024-09-19T05:40:10.587231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591358+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:40:10.591358+0000","last_peered":"2024-09-19T05:40:10.591358+0000","last_clean":"2024-09-19T05:40:10.591358+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:40:10.591358+0000","last_undegraded":"2024-09-19T05:40:10.591358+0000","last_fullsized":"2024-09-19T05:40:10.591358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589448+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:40:10.589448+0000","last_peered":"2024-09-19T05:40:10.589448+0000","last_clean":"2024-09-19T05:40:10.589448+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:40:10.589448+0000","last_undegraded":"2024-09-19T05:40:10.589448+0000","last_fullsized":"2024-09-19T05:40:10.589448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:41.192781+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:40:41.192781+0000","last_peered":"2024-09-19T05:40:41.192781+0000","last_clean":"2024-09-19T05:40:41.192781+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:40:41.192781+0000","last_undegraded":"2024-09-19T05:40:41.192781+0000","last_fullsized":"2024-09-19T05:40:41.192781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64622,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":122,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585946+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:40:10.585946+0000","last_peered":"2024-09-19T05:40:10.585946+0000","last_clean":"2024-09-19T05:40:10.585946+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:40:10.585946+0000","last_undegraded":"2024-09-19T05:40:10.585946+0000","last_fullsized":"2024-09-19T05:40:10.585946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589740+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:40:10.589740+0000","last_peered":"2024-09-19T05:40:10.589740+0000","last_clean":"2024-09-19T05:40:10.589740+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:40:10.589740+0000","last_undegraded":"2024-09-19T05:40:10.589740+0000","last_fullsized":"2024-09-19T05:40:10.589740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592349+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:40:10.592349+0000","last_peered":"2024-09-19T05:40:10.592349+0000","last_clean":"2024-09-19T05:40:10.592349+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:40:10.592349+0000","last_undegraded":"2024-09-19T05:40:10.592349+0000","last_fullsized":"2024-09-19T05:40:10.592349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590509+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:40:10.590509+0000","last_peered":"2024-09-19T05:40:10.590509+0000","last_clean":"2024-09-19T05:40:10.590509+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:40:10.590509+0000","last_undegraded":"2024-09-19T05:40:10.590509+0000","last_fullsized":"2024-09-19T05:40:10.590509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892943+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:40:10.892943+0000","last_peered":"2024-09-19T05:40:10.892943+0000","last_clean":"2024-09-19T05:40:10.892943+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:40:10.892943+0000","last_undegraded":"2024-09-19T05:40:10.892943+0000","last_fullsized":"2024-09-19T05:40:10.892943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590587+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:40:10.590587+0000","last_peered":"2024-09-19T05:40:10.590587+0000","last_clean":"2024-09-19T05:40:10.590587+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:40:10.590587+0000","last_undegraded":"2024-09-19T05:40:10.590587+0000","last_fullsized":"2024-09-19T05:40:10.590587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.893889+0000","last_change":"2024-09-19T05:40:10.893405+0000","last_active":"2024-09-19T05:40:10.893889+0000","last_peered":"2024-09-19T05:40:10.893889+0000","last_clean":"2024-09-19T05:40:10.893889+0000","last_became_active":"2024-09-19T05:40:10.892932+0000","last_became_peered":"2024-09-19T05:40:10.892932+0000","last_unstale":"2024-09-19T05:40:10.893889+0000","last_undegraded":"2024-09-19T05:40:10.893889+0000","last_fullsized":"2024-09-19T05:40:10.893889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:56.153847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591990+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:40:10.591990+0000","last_peered":"2024-09-19T05:40:10.591990+0000","last_clean":"2024-09-19T05:40:10.591990+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:40:10.591990+0000","last_undegraded":"2024-09-19T05:40:10.591990+0000","last_fullsized":"2024-09-19T05:40:10.591990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:41.186745+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:40:41.186745+0000","last_peered":"2024-09-19T05:40:41.186745+0000","last_clean":"2024-09-19T05:40:41.186745+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:40:41.186745+0000","last_undegraded":"2024-09-19T05:40:41.186745+0000","last_fullsized":"2024-09-19T05:40:41.186745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593701+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:40:10.593701+0000","last_peered":"2024-09-19T05:40:10.593701+0000","last_clean":"2024-09-19T05:40:10.593701+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:40:10.593701+0000","last_undegraded":"2024-09-19T05:40:10.593701+0000","last_fullsized":"2024-09-19T05:40:10.593701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892641+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:40:10.892641+0000","last_peered":"2024-09-19T05:40:10.892641+0000","last_clean":"2024-09-19T05:40:10.892641+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:40:10.892641+0000","last_undegraded":"2024-09-19T05:40:10.892641+0000","last_fullsized":"2024-09-19T05:40:10.892641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.585692+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:40:10.585692+0000","last_peered":"2024-09-19T05:40:10.585692+0000","last_clean":"2024-09-19T05:40:10.585692+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:40:10.585692+0000","last_undegraded":"2024-09-19T05:40:10.585692+0000","last_fullsized":"2024-09-19T05:40:10.585692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590988+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:40:10.590988+0000","last_peered":"2024-09-19T05:40:10.590988+0000","last_clean":"2024-09-19T05:40:10.590988+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:40:10.590988+0000","last_undegraded":"2024-09-19T05:40:10.590988+0000","last_fullsized":"2024-09-19T05:40:10.590988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594470+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:40:10.594470+0000","last_peered":"2024-09-19T05:40:10.594470+0000","last_clean":"2024-09-19T05:40:10.594470+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:40:10.594470+0000","last_undegraded":"2024-09-19T05:40:10.594470+0000","last_fullsized":"2024-09-19T05:40:10.594470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594185+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:40:10.594185+0000","last_peered":"2024-09-19T05:40:10.594185+0000","last_clean":"2024-09-19T05:40:10.594185+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:40:10.594185+0000","last_undegraded":"2024-09-19T05:40:10.594185+0000","last_fullsized":"2024-09-19T05:40:10.594185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594828+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:40:10.594828+0000","last_peered":"2024-09-19T05:40:10.594828+0000","last_clean":"2024-09-19T05:40:10.594828+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:40:10.594828+0000","last_undegraded":"2024-09-19T05:40:10.594828+0000","last_fullsized":"2024-09-19T05:40:10.594828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587999+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:40:10.587999+0000","last_peered":"2024-09-19T05:40:10.587999+0000","last_clean":"2024-09-19T05:40:10.587999+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:40:10.587999+0000","last_undegraded":"2024-09-19T05:40:10.587999+0000","last_fullsized":"2024-09-19T05:40:10.587999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591660+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:40:10.591660+0000","last_peered":"2024-09-19T05:40:10.591660+0000","last_clean":"2024-09-19T05:40:10.591660+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:40:10.591660+0000","last_undegraded":"2024-09-19T05:40:10.591660+0000","last_fullsized":"2024-09-19T05:40:10.591660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.596620+0000","last_change":"2024-09-19T05:40:10.596620+0000","last_active":"2024-09-19T05:40:10.596620+0000","last_peered":"2024-09-19T05:40:10.596620+0000","last_clean":"2024-09-19T05:40:10.596620+0000","last_became_active":"2024-09-19T05:40:10.596126+0000","last_became_peered":"2024-09-19T05:40:10.596126+0000","last_unstale":"2024-09-19T05:40:10.596620+0000","last_undegraded":"2024-09-19T05:40:10.596620+0000","last_fullsized":"2024-09-19T05:40:10.596620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:16.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:41.187266+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:40:41.187266+0000","last_peered":"2024-09-19T05:40:41.187266+0000","last_clean":"2024-09-19T05:40:41.187266+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:40:41.187266+0000","last_undegraded":"2024-09-19T05:40:41.187266+0000","last_fullsized":"2024-09-19T05:40:41.187266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590527+0000","last_change":"2024-09-19T05:40:10.590527+0000","last_active":"2024-09-19T05:40:10.590527+0000","last_peered":"2024-09-19T05:40:10.590527+0000","last_clean":"2024-09-19T05:40:10.590527+0000","last_became_active":"2024-09-19T05:40:10.590181+0000","last_became_peered":"2024-09-19T05:40:10.590181+0000","last_unstale":"2024-09-19T05:40:10.590527+0000","last_undegraded":"2024-09-19T05:40:10.590527+0000","last_fullsized":"2024-09-19T05:40:10.590527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:22:55.157416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587504+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:40:10.587504+0000","last_peered":"2024-09-19T05:40:10.587504+0000","last_clean":"2024-09-19T05:40:10.587504+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:40:10.587504+0000","last_undegraded":"2024-09-19T05:40:10.587504+0000","last_fullsized":"2024-09-19T05:40:10.587504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593931+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:40:10.593931+0000","last_peered":"2024-09-19T05:40:10.593931+0000","last_clean":"2024-09-19T05:40:10.593931+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:40:10.593931+0000","last_undegraded":"2024-09-19T05:40:10.593931+0000","last_fullsized":"2024-09-19T05:40:10.593931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587443+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:40:10.587443+0000","last_peered":"2024-09-19T05:40:10.587443+0000","last_clean":"2024-09-19T05:40:10.587443+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:40:10.587443+0000","last_undegraded":"2024-09-19T05:40:10.587443+0000","last_fullsized":"2024-09-19T05:40:10.587443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.893573+0000","last_change":"2024-09-19T05:40:10.893573+0000","last_active":"2024-09-19T05:40:10.893573+0000","last_peered":"2024-09-19T05:40:10.893573+0000","last_clean":"2024-09-19T05:40:10.893573+0000","last_became_active":"2024-09-19T05:40:10.893216+0000","last_became_peered":"2024-09-19T05:40:10.893216+0000","last_unstale":"2024-09-19T05:40:10.893573+0000","last_undegraded":"2024-09-19T05:40:10.893573+0000","last_fullsized":"2024-09-19T05:40:10.893573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:50.147170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892318+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:40:10.892318+0000","last_peered":"2024-09-19T05:40:10.892318+0000","last_clean":"2024-09-19T05:40:10.892318+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:40:10.892318+0000","last_undegraded":"2024-09-19T05:40:10.892318+0000","last_fullsized":"2024-09-19T05:40:10.892318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595266+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:40:10.595266+0000","last_peered":"2024-09-19T05:40:10.595266+0000","last_clean":"2024-09-19T05:40:10.595266+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:40:10.595266+0000","last_undegraded":"2024-09-19T05:40:10.595266+0000","last_fullsized":"2024-09-19T05:40:10.595266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594480+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:40:10.594480+0000","last_peered":"2024-09-19T05:40:10.594480+0000","last_clean":"2024-09-19T05:40:10.594480+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:40:10.594480+0000","last_undegraded":"2024-09-19T05:40:10.594480+0000","last_fullsized":"2024-09-19T05:40:10.594480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588192+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:40:10.588192+0000","last_peered":"2024-09-19T05:40:10.588192+0000","last_clean":"2024-09-19T05:40:10.588192+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:40:10.588192+0000","last_undegraded":"2024-09-19T05:40:10.588192+0000","last_fullsized":"2024-09-19T05:40:10.588192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594240+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:40:10.594240+0000","last_peered":"2024-09-19T05:40:10.594240+0000","last_clean":"2024-09-19T05:40:10.594240+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:40:10.594240+0000","last_undegraded":"2024-09-19T05:40:10.594240+0000","last_fullsized":"2024-09-19T05:40:10.594240+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.590867+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:40:10.590867+0000","last_peered":"2024-09-19T05:40:10.590867+0000","last_clean":"2024-09-19T05:40:10.590867+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:40:10.590867+0000","last_undegraded":"2024-09-19T05:40:10.590867+0000","last_fullsized":"2024-09-19T05:40:10.590867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594398+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:40:10.594398+0000","last_peered":"2024-09-19T05:40:10.594398+0000","last_clean":"2024-09-19T05:40:10.594398+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:40:10.594398+0000","last_undegraded":"2024-09-19T05:40:10.594398+0000","last_fullsized":"2024-09-19T05:40:10.594398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591310+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:40:10.591310+0000","last_peered":"2024-09-19T05:40:10.591310+0000","last_clean":"2024-09-19T05:40:10.591310+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:40:10.591310+0000","last_undegraded":"2024-09-19T05:40:10.591310+0000","last_fullsized":"2024-09-19T05:40:10.591310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595203+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:40:10.595203+0000","last_peered":"2024-09-19T05:40:10.595203+0000","last_clean":"2024-09-19T05:40:10.595203+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:40:10.595203+0000","last_undegraded":"2024-09-19T05:40:10.595203+0000","last_fullsized":"2024-09-19T05:40:10.595203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594790+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:40:10.594790+0000","last_peered":"2024-09-19T05:40:10.594790+0000","last_clean":"2024-09-19T05:40:10.594790+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:40:10.594790+0000","last_undegraded":"2024-09-19T05:40:10.594790+0000","last_fullsized":"2024-09-19T05:40:10.594790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587984+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:40:10.587984+0000","last_peered":"2024-09-19T05:40:10.587984+0000","last_clean":"2024-09-19T05:40:10.587984+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:40:10.587984+0000","last_undegraded":"2024-09-19T05:40:10.587984+0000","last_fullsized":"2024-09-19T05:40:10.587984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.586387+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:40:10.586387+0000","last_peered":"2024-09-19T05:40:10.586387+0000","last_clean":"2024-09-19T05:40:10.586387+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:40:10.586387+0000","last_undegraded":"2024-09-19T05:40:10.586387+0000","last_fullsized":"2024-09-19T05:40:10.586387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594933+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:40:10.594933+0000","last_peered":"2024-09-19T05:40:10.594933+0000","last_clean":"2024-09-19T05:40:10.594933+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:40:10.594933+0000","last_undegraded":"2024-09-19T05:40:10.594933+0000","last_fullsized":"2024-09-19T05:40:10.594933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589559+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:40:10.589559+0000","last_peered":"2024-09-19T05:40:10.589559+0000","last_clean":"2024-09-19T05:40:10.589559+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:40:10.589559+0000","last_undegraded":"2024-09-19T05:40:10.589559+0000","last_fullsized":"2024-09-19T05:40:10.589559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.891939+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:40:10.891939+0000","last_peered":"2024-09-19T05:40:10.891939+0000","last_clean":"2024-09-19T05:40:10.891939+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:40:10.891939+0000","last_undegraded":"2024-09-19T05:40:10.891939+0000","last_fullsized":"2024-09-19T05:40:10.891939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892024+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:40:10.892024+0000","last_peered":"2024-09-19T05:40:10.892024+0000","last_clean":"2024-09-19T05:40:10.892024+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:40:10.892024+0000","last_undegraded":"2024-09-19T05:40:10.892024+0000","last_fullsized":"2024-09-19T05:40:10.892024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.595181+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:40:10.595181+0000","last_peered":"2024-09-19T05:40:10.595181+0000","last_clean":"2024-09-19T05:40:10.595181+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:40:10.595181+0000","last_undegraded":"2024-09-19T05:40:10.595181+0000","last_fullsized":"2024-09-19T05:40:10.595181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591269+0000","last_change":"2024-09-19T05:40:10.590802+0000","last_active":"2024-09-19T05:40:10.591269+0000","last_peered":"2024-09-19T05:40:10.591269+0000","last_clean":"2024-09-19T05:40:10.591269+0000","last_became_active":"2024-09-19T05:40:10.590515+0000","last_became_peered":"2024-09-19T05:40:10.590515+0000","last_unstale":"2024-09-19T05:40:10.591269+0000","last_undegraded":"2024-09-19T05:40:10.591269+0000","last_fullsized":"2024-09-19T05:40:10.591269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:37.010052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.591075+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:40:10.591075+0000","last_peered":"2024-09-19T05:40:10.591075+0000","last_clean":"2024-09-19T05:40:10.591075+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:40:10.591075+0000","last_undegraded":"2024-09-19T05:40:10.591075+0000","last_fullsized":"2024-09-19T05:40:10.591075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589178+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:40:10.589178+0000","last_peered":"2024-09-19T05:40:10.589178+0000","last_clean":"2024-09-19T05:40:10.589178+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:40:10.589178+0000","last_undegraded":"2024-09-19T05:40:10.589178+0000","last_fullsized":"2024-09-19T05:40:10.589178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.592427+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:40:10.592427+0000","last_peered":"2024-09-19T05:40:10.592427+0000","last_clean":"2024-09-19T05:40:10.592427+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:40:10.592427+0000","last_undegraded":"2024-09-19T05:40:10.592427+0000","last_fullsized":"2024-09-19T05:40:10.592427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.593998+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:40:10.593998+0000","last_peered":"2024-09-19T05:40:10.593998+0000","last_clean":"2024-09-19T05:40:10.593998+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:40:10.593998+0000","last_undegraded":"2024-09-19T05:40:10.593998+0000","last_fullsized":"2024-09-19T05:40:10.593998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587111+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:40:10.587111+0000","last_peered":"2024-09-19T05:40:10.587111+0000","last_clean":"2024-09-19T05:40:10.587111+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:40:10.587111+0000","last_undegraded":"2024-09-19T05:40:10.587111+0000","last_fullsized":"2024-09-19T05:40:10.587111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.587886+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:40:10.587886+0000","last_peered":"2024-09-19T05:40:10.587886+0000","last_clean":"2024-09-19T05:40:10.587886+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:40:10.587886+0000","last_undegraded":"2024-09-19T05:40:10.587886+0000","last_fullsized":"2024-09-19T05:40:10.587886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.589254+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:40:10.589254+0000","last_peered":"2024-09-19T05:40:10.589254+0000","last_clean":"2024-09-19T05:40:10.589254+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:40:10.589254+0000","last_undegraded":"2024-09-19T05:40:10.589254+0000","last_fullsized":"2024-09-19T05:40:10.589254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.588920+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:40:10.588920+0000","last_peered":"2024-09-19T05:40:10.588920+0000","last_clean":"2024-09-19T05:40:10.588920+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:40:10.588920+0000","last_undegraded":"2024-09-19T05:40:10.588920+0000","last_fullsized":"2024-09-19T05:40:10.588920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892539+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:40:10.892539+0000","last_peered":"2024-09-19T05:40:10.892539+0000","last_clean":"2024-09-19T05:40:10.892539+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:40:10.892539+0000","last_undegraded":"2024-09-19T05:40:10.892539+0000","last_fullsized":"2024-09-19T05:40:10.892539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.892708+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:40:10.892708+0000","last_peered":"2024-09-19T05:40:10.892708+0000","last_clean":"2024-09-19T05:40:10.892708+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:40:10.892708+0000","last_undegraded":"2024-09-19T05:40:10.892708+0000","last_fullsized":"2024-09-19T05:40:10.892708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-19T05:40:10.594729+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:40:10.594729+0000","last_peered":"2024-09-19T05:40:10.594729+0000","last_clean":"2024-09-19T05:40:10.594729+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:40:10.594729+0000","last_undegraded":"2024-09-19T05:40:10.594729+0000","last_fullsized":"2024-09-19T05:40:10.594729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72616,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":585,"num_read_kb":370,"num_write":252,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547232,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":499829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1151267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939692,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":503462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168625,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1036288,"data_stored":499744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1634304,"data_stored":1097029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":495263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1091825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":499337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":494578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1105920,"data_stored":561528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1085440,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":496719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:40:44.344 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:40:44.344 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T05:40:44.344 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T05:40:44.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T05:40:44.612 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:45.020 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:44 smithi038 ceph-mon[31227]: pgmap v336: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:45.020 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:44 smithi038 ceph-mon[31227]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 -- 172.21.15.38:0/3220102405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670108300 msgr2=0x7f667010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 --2- 172.21.15.38:0/3220102405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670108300 0x7f667010ec70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6668009930 tx=0x7f666802a420 comp rx=0 tx=0).stop 2024-09-19T05:40:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 -- 172.21.15.38:0/3220102405 shutdown_connections 2024-09-19T05:40:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 --2- 172.21.15.38:0/3220102405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670108300 0x7f667010ec70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 --2- 172.21.15.38:0/3220102405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670103780 0x7f6670107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 --2- 172.21.15.38:0/3220102405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670102dd0 0x7f66701031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 -- 172.21.15.38:0/3220102405 >> 172.21.15.38:0/3220102405 conn(0x7f66700fc7d0 msgr2=0x7f66700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 -- 172.21.15.38:0/3220102405 shutdown_connections 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.100+0000 7f6676e46640 1 -- 172.21.15.38:0/3220102405 wait complete. 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 Processor -- start 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 -- start start 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670102dd0 0x7f6670198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670108300 0x7f667019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6670112120 con 0x7f6670103780 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6670111fa0 con 0x7f6670102dd0 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6676e46640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66701122a0 con 0x7f6670108300 2024-09-19T05:40:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42898/0 (socket says 172.21.15.38:42898) 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 -- 172.21.15.38:0/1974327394 learned_addr learned my addr 172.21.15.38:0/1974327394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f66753e3640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670108300 0x7f667019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f6674be2640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670102dd0 0x7f6670198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 -- 172.21.15.38:0/1974327394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670108300 msgr2=0x7f667019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670108300 0x7f667019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 -- 172.21.15.38:0/1974327394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670102dd0 msgr2=0x7f6670198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670102dd0 0x7f6670198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.101+0000 7f666ffff640 1 -- 172.21.15.38:0/1974327394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667019d510 con 0x7f6670103780 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f666ffff640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f665800d950 tx=0x7f665800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658017070 con 0x7f6670103780 2024-09-19T05:40:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6658009c30 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667019d800 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580047a0 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.102+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66701a50e0 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.103+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66701031b0 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.104+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6658004020 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.105+0000 7f666dffb640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 0x7f665c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.105+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f66580cf990 con 0x7f6670103780 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.105+0000 7f6674be2640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 0x7f665c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.105+0000 7f6674be2640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 0x7f665c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6660005fd0 tx=0x7f6660009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.109+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6658020020 con 0x7f6670103780 2024-09-19T05:40:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:44 smithi179 ceph-mon[35781]: pgmap v336: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:44 smithi179 ceph-mon[35781]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:45.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:44 smithi047 ceph-mon[35327]: pgmap v336: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:45.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:44 smithi047 ceph-mon[35327]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.323+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f667010a2c0 con 0x7f6670103780 2024-09-19T05:40:45.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.324+0000 7f666dffb640 1 -- 172.21.15.38:0/1974327394 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f665809c190 con 0x7f6670103780 2024-09-19T05:40:45.326 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:45.326 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 msgr2=0x7f665c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 0x7f665c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6660005fd0 tx=0x7f6660009760 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 msgr2=0x7f6670198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f665800d950 tx=0x7f665800de10 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 shutdown_connections 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f665c0a32d0 0x7f665c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6670108300 0x7f667019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6670103780 0x7f6670198aa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 --2- 172.21.15.38:0/1974327394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6670102dd0 0x7f6670198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:45.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 >> 172.21.15.38:0/1974327394 conn(0x7f66700fc7d0 msgr2=0x7f66701063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 shutdown_connections 2024-09-19T05:40:45.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:45.327+0000 7f6676e46640 1 -- 172.21.15.38:0/1974327394 wait complete. 2024-09-19T05:40:45.972 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T05:40:45.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:40:46.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:45 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1974327394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:40:46.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:45 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1974327394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:40:46.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:45 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1974327394' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:40:46.230 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:46.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 -- 172.21.15.38:0/47976081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28100a60 msgr2=0x7f4f28100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 --2- 172.21.15.38:0/47976081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28100a60 0x7f4f28100ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009930 tx=0x7f4f1802a420 comp rx=0 tx=0).stop 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 -- 172.21.15.38:0/47976081 shutdown_connections 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 --2- 172.21.15.38:0/47976081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28101400 0x7f4f2810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 --2- 172.21.15.38:0/47976081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28100a60 0x7f4f28100ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 --2- 172.21.15.38:0/47976081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f28106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 -- 172.21.15.38:0/47976081 >> 172.21.15.38:0/47976081 conn(0x7f4f280fc780 msgr2=0x7f4f280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 -- 172.21.15.38:0/47976081 shutdown_connections 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 -- 172.21.15.38:0/47976081 wait complete. 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.726+0000 7f4f3015b640 1 Processor -- start 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 -- start start 2024-09-19T05:40:46.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 0x7f4f28198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f2819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f28110e60 con 0x7f4f28101400 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4f28110ce0 con 0x7f4f28106a60 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f3015b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f28110fe0 con 0x7f4f28100a60 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42918/0 (socket says 172.21.15.38:42918) 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 -- 172.21.15.38:0/732747436 learned_addr learned my addr 172.21.15.38:0/732747436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2def7640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 0x7f4f28198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2e6f8640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f2819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 -- 172.21.15.38:0/732747436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 msgr2=0x7f4f28198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 0x7f4f28198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 -- 172.21.15.38:0/732747436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 msgr2=0x7f4f2819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f2819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 -- 172.21.15.38:0/732747436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2819d410 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2e6f8640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f2819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2def7640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 0x7f4f28198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.727+0000 7f4f2d6f6640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009230 tx=0x7f4f18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.728+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1803f030 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.728+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f18003ad0 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.728+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2819d6a0 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.728+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18035070 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.728+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f281a4fe0 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.729+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f281021a0 con 0x7f4f28101400 2024-09-19T05:40:46.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.730+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4f18042030 con 0x7f4f28101400 2024-09-19T05:40:46.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.730+0000 7f4f1effd640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 0x7f4f140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:46.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.731+0000 7f4f2def7640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 0x7f4f140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:46.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.731+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4f180f0310 con 0x7f4f28101400 2024-09-19T05:40:46.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.731+0000 7f4f2def7640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 0x7f4f140a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005fd0 tx=0x7f4f24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:46.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.735+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f180bcc10 con 0x7f4f28101400 2024-09-19T05:40:46.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.955+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f28109fe0 con 0x7f4f28101400 2024-09-19T05:40:46.957 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:46 smithi038 ceph-mon[31227]: pgmap v337: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.955+0000 7f4f1effd640 1 -- 172.21.15.38:0/732747436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7f4f180bc510 con 0x7f4f28101400 2024-09-19T05:40:46.957 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:46.958 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:40:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 msgr2=0x7f4f140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 0x7f4f140a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005fd0 tx=0x7f4f24005850 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 msgr2=0x7f4f281989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009230 tx=0x7f4f18004270 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 shutdown_connections 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4f140a32d0 0x7f4f140a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f28106a60 0x7f4f2819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f28101400 0x7f4f281989a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 --2- 172.21.15.38:0/732747436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f28100a60 0x7f4f28198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 >> 172.21.15.38:0/732747436 conn(0x7f4f280fc780 msgr2=0x7f4f2806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 shutdown_connections 2024-09-19T05:40:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:46.958+0000 7f4f3015b640 1 -- 172.21.15.38:0/732747436 wait complete. 2024-09-19T05:40:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:46 smithi179 ceph-mon[35781]: pgmap v337: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:47.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:46 smithi047 ceph-mon[35327]: pgmap v337: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:47.721 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:40:47.984 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:48.105 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:47 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/732747436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:48.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:47 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/732747436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:48.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:47 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/732747436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 -- 172.21.15.38:0/2419028795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 msgr2=0x7f2bdc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 --2- 172.21.15.38:0/2419028795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc10ea90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009900 tx=0x7f2bd402a410 comp rx=0 tx=0).stop 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 -- 172.21.15.38:0/2419028795 shutdown_connections 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 --2- 172.21.15.38:0/2419028795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc10ea90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 --2- 172.21.15.38:0/2419028795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc102790 0x7f2bdc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 --2- 172.21.15.38:0/2419028795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 -- 172.21.15.38:0/2419028795 >> 172.21.15.38:0/2419028795 conn(0x7f2bdc0fd8c0 msgr2=0x7f2bdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 -- 172.21.15.38:0/2419028795 shutdown_connections 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.470+0000 7f2be08c4640 1 -- 172.21.15.38:0/2419028795 wait complete. 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 Processor -- start 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 -- start start 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:48.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc102790 0x7f2bdc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111f70 con 0x7f2bdc106f10 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111df0 con 0x7f2bdc101de0 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.471+0000 7f2be08c4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1120f0 con 0x7f2bdc102790 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39958/0 (socket says 172.21.15.38:39958) 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 -- 172.21.15.38:0/1574920354 learned_addr learned my addr 172.21.15.38:0/1574920354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bda59c640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:48.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bd9d9b640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc102790 0x7f2bdc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 -- 172.21.15.38:0/1574920354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc102790 msgr2=0x7f2bdc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc102790 0x7f2bdc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 -- 172.21.15.38:0/1574920354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 msgr2=0x7f2bdc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 -- 172.21.15.38:0/1574920354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc19d360 con 0x7f2bdc106f10 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bda59c640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bdad9d640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005ab0 tx=0x7f2bd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd403f030 con 0x7f2bdc106f10 2024-09-19T05:40:48.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd4003ad0 con 0x7f2bdc106f10 2024-09-19T05:40:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.472+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc19d5f0 con 0x7f2bdc106f10 2024-09-19T05:40:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.473+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4035070 con 0x7f2bdc106f10 2024-09-19T05:40:48.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.473+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a4f30 con 0x7f2bdc106f10 2024-09-19T05:40:48.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.474+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc1032c0 con 0x7f2bdc106f10 2024-09-19T05:40:48.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.475+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2bd4042030 con 0x7f2bdc106f10 2024-09-19T05:40:48.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.476+0000 7f2bd37fe640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 0x7f2bc80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:48.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.476+0000 7f2bda59c640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 0x7f2bc80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:48.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.476+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2bd40f0700 con 0x7f2bdc106f10 2024-09-19T05:40:48.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.476+0000 7f2bda59c640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 0x7f2bc80a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009fd0 tx=0x7f2bcc009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:48.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.480+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bd40bc8b0 con 0x7f2bdc106f10 2024-09-19T05:40:48.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.693+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bdc10a0f0 con 0x7f2bdc106f10 2024-09-19T05:40:48.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.694+0000 7f2bd37fe640 1 -- 172.21.15.38:0/1574920354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7f2bd40c0bb0 con 0x7f2bdc106f10 2024-09-19T05:40:48.696 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:48.696 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:40:48.696 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 msgr2=0x7f2bc80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 0x7f2bc80a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009fd0 tx=0x7f2bcc009760 comp rx=0 tx=0).stop 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 msgr2=0x7f2bdc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005ab0 tx=0x7f2bd4004270 comp rx=0 tx=0).stop 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 shutdown_connections 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2bc80a32d0 0x7f2bc80a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bdc106f10 0x7f2bdc19cc80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bdc102790 0x7f2bdc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 --2- 172.21.15.38:0/1574920354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bdc101de0 0x7f2bdc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.696+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 >> 172.21.15.38:0/1574920354 conn(0x7f2bdc0fd8c0 msgr2=0x7f2bdc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.697+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 shutdown_connections 2024-09-19T05:40:48.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:48.697+0000 7f2be08c4640 1 -- 172.21.15.38:0/1574920354 wait complete. 2024-09-19T05:40:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:48 smithi038 ceph-mon[31227]: pgmap v338: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:49.070 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:48 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1574920354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:49.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:48 smithi179 ceph-mon[35781]: pgmap v338: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:49.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:48 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1574920354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:49.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:48 smithi047 ceph-mon[35327]: pgmap v338: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:49.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:48 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1574920354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:49.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:40:49.760 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 -- 172.21.15.38:0/2861577607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698100ca0 msgr2=0x7fa698106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 --2- 172.21.15.38:0/2861577607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698100ca0 0x7fa698106840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009930 tx=0x7fa68c02a420 comp rx=0 tx=0).stop 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 -- 172.21.15.38:0/2861577607 shutdown_connections 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 --2- 172.21.15.38:0/2861577607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa698106f50 0x7fa69810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 --2- 172.21.15.38:0/2861577607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698100ca0 0x7fa698106840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 --2- 172.21.15.38:0/2861577607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6981002f0 0x7fa6981006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 -- 172.21.15.38:0/2861577607 >> 172.21.15.38:0/2861577607 conn(0x7fa698077210 msgr2=0x7fa6980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 -- 172.21.15.38:0/2861577607 shutdown_connections 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.256+0000 7fa69d167640 1 -- 172.21.15.38:0/2861577607 wait complete. 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 Processor -- start 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 -- start start 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 0x7fa6981984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa698100ca0 0x7fa698198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa698111fa0 con 0x7fa698106f50 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa698111e20 con 0x7fa6981002f0 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69d167640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa698112120 con 0x7fa698100ca0 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:50.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 0x7fa6981984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39968/0 (socket says 172.21.15.38:39968) 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69759e640 1 -- 172.21.15.38:0/980350345 learned_addr learned my addr 172.21.15.38:0/980350345 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa696d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 0x7fa6981984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:54418/0 (socket says 172.21.15.38:54418) 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.257+0000 7fa69659c640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa698100ca0 0x7fa698198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 -- 172.21.15.38:0/980350345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa698100ca0 msgr2=0x7fa698198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa698100ca0 0x7fa698198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 -- 172.21.15.38:0/980350345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 msgr2=0x7fa6981984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 0x7fa6981984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 -- 172.21.15.38:0/980350345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69819d490 con 0x7fa698106f50 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69759e640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa69000eaa0 tx=0x7fa69000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000cdb0 con 0x7fa698106f50 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69819d780 con 0x7fa698106f50 2024-09-19T05:40:50.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690005020 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900116b0 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.258+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a5060 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.259+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6981006d0 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.261+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa690005560 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.261+0000 7fa67ffff640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 0x7fa6800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.261+0000 7fa696d9d640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 0x7fa6800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.261+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6900cff40 con 0x7fa698106f50 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.262+0000 7fa696d9d640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 0x7fa6800a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa68800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:50.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.265+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa690016030 con 0x7fa698106f50 2024-09-19T05:40:50.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.445+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa698109b90 con 0x7fa698106f50 2024-09-19T05:40:50.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.448+0000 7fa67ffff640 1 -- 172.21.15.38:0/980350345 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24660 (secure 0 0 0) 0x7fa69009c150 con 0x7fa698106f50 2024-09-19T05:40:50.450 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:50.450 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":89,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:40:10.573456+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:40:50.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 msgr2=0x7fa6800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 0x7fa6800a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa688005fd0 tx=0x7fa68800b390 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 msgr2=0x7fa69819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa69000eaa0 tx=0x7fa69000ef60 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 shutdown_connections 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa6800a32d0 0x7fa6800a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa698106f50 0x7fa69819cdb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa698100ca0 0x7fa698198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 --2- 172.21.15.38:0/980350345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981002f0 0x7fa6981984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 >> 172.21.15.38:0/980350345 conn(0x7fa698077210 msgr2=0x7fa6981052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 shutdown_connections 2024-09-19T05:40:50.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:50.451+0000 7fa69d167640 1 -- 172.21.15.38:0/980350345 wait complete. 2024-09-19T05:40:51.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:50 smithi038 ceph-mon[31227]: pgmap v339: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:51.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:50 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/980350345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:51.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:50 smithi179 ceph-mon[35781]: pgmap v339: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:51.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:50 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/980350345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:51.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:40:51.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:50 smithi047 ceph-mon[35327]: pgmap v339: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:51.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:50 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/980350345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 -- 172.21.15.38:0/4168267315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0106a80 msgr2=0x7f6fa0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 --2- 172.21.15.38:0/4168267315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0106a80 0x7f6fa0106e60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009930 tx=0x7f6f9002a420 comp rx=0 tx=0).stop 2024-09-19T05:40:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 -- 172.21.15.38:0/4168267315 shutdown_connections 2024-09-19T05:40:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 --2- 172.21.15.38:0/4168267315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0101420 0x7f6fa010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 --2- 172.21.15.38:0/4168267315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0100a80 0x7f6fa0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 --2- 172.21.15.38:0/4168267315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0106a80 0x7f6fa0106e60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 -- 172.21.15.38:0/4168267315 >> 172.21.15.38:0/4168267315 conn(0x7f6fa00fc7a0 msgr2=0x7f6fa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 -- 172.21.15.38:0/4168267315 shutdown_connections 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.927+0000 7f6fa577b640 1 -- 172.21.15.38:0/4168267315 wait complete. 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.928+0000 7f6fa577b640 1 Processor -- start 2024-09-19T05:40:51.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 -- start start 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0101420 0x7f6fa01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0106a80 0x7f6fa01a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0110ed0 con 0x7f6fa0100a80 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0110d50 con 0x7f6fa0101420 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6fa577b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0111050 con 0x7f6fa0106a80 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:51.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39990/0 (socket says 172.21.15.38:39990) 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 -- 172.21.15.38:0/2698020719 learned_addr learned my addr 172.21.15.38:0/2698020719 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9e7fc640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0101420 0x7f6fa01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9f7fe640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0106a80 0x7f6fa01a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 -- 172.21.15.38:0/2698020719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0106a80 msgr2=0x7f6fa01a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0106a80 0x7f6fa01a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 -- 172.21.15.38:0/2698020719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0101420 msgr2=0x7f6fa01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0101420 0x7f6fa01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.929+0000 7f6f9effd640 1 -- 172.21.15.38:0/2698020719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa01a4820 con 0x7f6fa0100a80 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6f9effd640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f9002a940 tx=0x7f6f90003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9003f070 con 0x7f6fa0100a80 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01a49f0 con 0x7f6fa0100a80 2024-09-19T05:40:51.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f90031c90 con 0x7f6fa0100a80 2024-09-19T05:40:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90035030 con 0x7f6fa0100a80 2024-09-19T05:40:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.930+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01a4e50 con 0x7f6fa0100a80 2024-09-19T05:40:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.931+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa0100ee0 con 0x7f6fa0100a80 2024-09-19T05:40:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.932+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6f90042070 con 0x7f6fa0100a80 2024-09-19T05:40:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.933+0000 7f6f87fff640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 0x7f6f880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.933+0000 7f6f9e7fc640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 0x7f6f880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.933+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6f900f0530 con 0x7f6fa0100a80 2024-09-19T05:40:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.933+0000 7f6f9e7fc640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 0x7f6f880a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006b2e0 tx=0x7f6f94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:51.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:51.937+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f900bc6e0 con 0x7f6fa0100a80 2024-09-19T05:40:52.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.157+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fa01a5020 con 0x7f6fa0100a80 2024-09-19T05:40:52.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.157+0000 7f6f87fff640 1 -- 172.21.15.38:0/2698020719 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7f6f900c09e0 con 0x7f6fa0100a80 2024-09-19T05:40:52.159 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:40:52.160 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:40:52.160 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:40:52.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 msgr2=0x7f6f880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 0x7f6f880a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006b2e0 tx=0x7f6f94005f90 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 msgr2=0x7f6fa0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f9002a940 tx=0x7f6f90003b90 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 shutdown_connections 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6f880a32d0 0x7f6f880a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fa0106a80 0x7f6fa01a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fa0101420 0x7f6fa01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 --2- 172.21.15.38:0/2698020719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fa0100a80 0x7f6fa0199260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 >> 172.21.15.38:0/2698020719 conn(0x7f6fa00fc7a0 msgr2=0x7f6fa00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 shutdown_connections 2024-09-19T05:40:52.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:52.160+0000 7f6fa577b640 1 -- 172.21.15.38:0/2698020719 wait complete. 2024-09-19T05:40:53.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:39:02.544314+0000', 'modified': '2024-09-19T05:39:48.102894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24496, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14547': {'gid': 14547, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/4064878755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 4064878755}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 4064878755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/2963619728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 2963619728}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 2963619728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6841/3698066275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 3698066275}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 3698066275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/3758113926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3758113926}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3758113926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/2514682242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2514682242}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 2514682242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.47:6837/2962930600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2962930600}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2962930600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.179:6835/994628887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 994628887}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 994628887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.179:6837/164318179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 164318179}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 164318179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.47:6839/1610791508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 1610791508}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 1610791508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.179:6839/2081935059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2081935059}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2081935059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24496, 24485, 24475, 24481, 24455]} 2024-09-19T05:40:53.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T05:40:53.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T05:40:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:52 smithi179 ceph-mon[35781]: pgmap v340: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:52 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2698020719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:52 smithi047 ceph-mon[35327]: pgmap v340: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2698020719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:53.304 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:53.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:52 smithi038 ceph-mon[31227]: pgmap v340: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:40:53.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:52 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2698020719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:40:53.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 -- 172.21.15.38:0/556576380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c0102da0 msgr2=0x7ff9c0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 --2- 172.21.15.38:0/556576380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c0102da0 0x7ff9c0103180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b002a480 comp rx=0 tx=0).stop 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 -- 172.21.15.38:0/556576380 shutdown_connections 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 --2- 172.21.15.38:0/556576380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 0x7ff9c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 --2- 172.21.15.38:0/556576380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c01036c0 0x7ff9c010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 --2- 172.21.15.38:0/556576380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c0102da0 0x7ff9c0103180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 -- 172.21.15.38:0/556576380 >> 172.21.15.38:0/556576380 conn(0x7ff9c00fc7a0 msgr2=0x7ff9c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 -- 172.21.15.38:0/556576380 shutdown_connections 2024-09-19T05:40:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.806+0000 7ff9c5733640 1 -- 172.21.15.38:0/556576380 wait complete. 2024-09-19T05:40:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 Processor -- start 2024-09-19T05:40:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 -- start start 2024-09-19T05:40:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c0102da0 0x7ff9c019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 0x7ff9c019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111fa0 con 0x7ff9c01036c0 2024-09-19T05:40:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111e20 con 0x7ff9c010c180 2024-09-19T05:40:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9c5733640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0112120 con 0x7ff9c0102da0 2024-09-19T05:40:53.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40006/0 (socket says 172.21.15.38:40006) 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9be7fc640 1 -- 172.21.15.38:0/912325366 learned_addr learned my addr 172.21.15.38:0/912325366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.807+0000 7ff9beffd640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c0102da0 0x7ff9c019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 0x7ff9c019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 -- 172.21.15.38:0/912325366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c0102da0 msgr2=0x7ff9c019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c0102da0 0x7ff9c019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 -- 172.21.15.38:0/912325366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 msgr2=0x7ff9c019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 0x7ff9c019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 -- 172.21.15.38:0/912325366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c019f6d0 con 0x7ff9c01036c0 2024-09-19T05:40:53.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b4a0 tx=0x7ff9b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9a7fff640 1 -- 172.21.15.38:0/912325366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4014070 con 0x7ff9c01036c0 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c019f9c0 con 0x7ff9c01036c0 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9a7fff640 1 -- 172.21.15.38:0/912325366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b4010af0 con 0x7ff9c01036c0 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9a7fff640 1 -- 172.21.15.38:0/912325366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4017d10 con 0x7ff9c01036c0 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.808+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a72a0 con 0x7ff9c01036c0 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 Processor -- start 2024-09-19T05:40:53.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9a7fff640 1 -- 172.21.15.38:0/912325366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff9b4017070 con 0x7ff9c01036c0 2024-09-19T05:40:53.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 -- start start 2024-09-19T05:40:53.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c82adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c837fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55866c791090 con 0x55866c82b2f0 2024-09-19T05:40:53.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55866c738a70 con 0x55866c82a9d0 2024-09-19T05:40:53.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55866c78d050 con 0x55866c835bb0 2024-09-19T05:40:53.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40008/0 (socket says 172.21.15.38:40008) 2024-09-19T05:40:53.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 -- 172.21.15.38:0/2700862580 learned_addr learned my addr 172.21.15.38:0/2700862580 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c82adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c837fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 -- 172.21.15.38:0/2700862580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 msgr2=0x55866c837fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c837fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 -- 172.21.15.38:0/2700862580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 msgr2=0x55866c82adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c82adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9beffd640 1 -- 172.21.15.38:0/2700862580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55866c745130 con 0x55866c82b2f0 2024-09-19T05:40:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.811+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c837fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9a7fff640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 0x7ff9a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c820b66196cbfa74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 0x7ff9a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9a7fff640 1 -- 172.21.15.38:0/912325366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff9b40cf0c0 con 0x7ff9c01036c0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9a57fa640 1 -- 172.21.15.38:0/2700862580 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0040070 con 0x55866c82b2f0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9a57fa640 1 -- 172.21.15.38:0/2700862580 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b0005020 con 0x55866c82b2f0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.812+0000 7ff9a57fa640 1 -- 172.21.15.38:0/2700862580 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0035040 con 0x55866c82b2f0 2024-09-19T05:40:53.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2700862580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 msgr2=0x55866c835670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 0x7ff9a80a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8009740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b0031630 comp rx=0 tx=0).stop 2024-09-19T05:40:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2700862580 shutdown_connections 2024-09-19T05:40:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c837fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c835670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2700862580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c82adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2700862580 >> 172.21.15.38:0/2700862580 conn(0x55866c826f60 msgr2=0x55866c827340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2700862580 shutdown_connections 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2700862580 wait complete. 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- start start 2024-09-19T05:40:53.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 Processor -- start 2024-09-19T05:40:53.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c82b2f0 0x55866c832710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55866c838780 con 0x55866c835bb0 2024-09-19T05:40:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55866c74bf20 con 0x55866c82a9d0 2024-09-19T05:40:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55866c838900 con 0x55866c82b2f0 2024-09-19T05:40:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.813+0000 7ff9c7b0c740 10 client.0 _ll_register_callbacks cb 0x55866c715030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40024/0 (socket says 172.21.15.38:40024) 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:54484/0 (socket says 172.21.15.38:54484) 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 -- 172.21.15.38:0/3647700133 learned_addr learned my addr 172.21.15.38:0/3647700133 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c82b2f0 0x55866c832710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 -- 172.21.15.38:0/3647700133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c82b2f0 msgr2=0x55866c832710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c82b2f0 0x55866c832710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 -- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 msgr2=0x55866c8353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 -- 172.21.15.38:0/3647700133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55866c5022b0 con 0x55866c835bb0 2024-09-19T05:40:53.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9beffd640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9b801e970 tx=0x7ff9b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00c92c032be6374 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8010d60 con 0x55866c835bb0 2024-09-19T05:40:53.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff9b801f420 con 0x55866c835bb0 2024-09-19T05:40:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.814+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b0005020 con 0x55866c835bb0 2024-09-19T05:40:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9b8004b20 con 0x55866c835bb0 2024-09-19T05:40:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55866c843fe0 con 0x55866c835bb0 2024-09-19T05:40:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b0005020 con 0x55866c835bb0 2024-09-19T05:40:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7ff9b800f040 con 0x55866c835bb0 2024-09-19T05:40:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:40:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 20 client.15063 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c846640 0x55866c847ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 4 client.15063 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55866c79e600 con 0x55866c846640 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.815+0000 7ff9beffd640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c846640 0x55866c847ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.816+0000 7ff9a2ff5640 10 client.15063 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.816+0000 7ff9beffd640 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c846640 0x55866c847ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d1aa8908df85304 server_cookie=66c09e8744ce99aa in_seq=0 out_seq=0 2024-09-19T05:40:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/3647700133 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55866c79e600 con 0x55866c846640 2024-09-19T05:40:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9a2ff5640 10 client.15063 handle_command_reply: tid=0 2024-09-19T05:40:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 client.15063 shutdown 2024-09-19T05:40:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 msgr2=0x55866c832c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9b801e970 tx=0x7ff9b8004a40 comp rx=0 tx=0).stop 2024-09-19T05:40:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 shutdown_connections 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c846640 0x55866c847ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c835bb0 0x55866c832c50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c82b2f0 0x55866c832710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/3647700133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 >> 172.21.15.38:0/3647700133 conn(0x55866c826f60 msgr2=0x55866c82cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9a2ff5640 0 client.15063 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:40:53.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 shutdown_connections 2024-09-19T05:40:53.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/3647700133 wait complete. 2024-09-19T05:40:53.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.817+0000 7ff9c7b0c740 20 client.15063 trim_cache size 0 max 16384 2024-09-19T05:40:53.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 Processor -- start 2024-09-19T05:40:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 -- start start 2024-09-19T05:40:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c842000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c845f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55866c845df0 con 0x55866c82b2f0 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55866c8464c0 con 0x55866c82a9d0 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55866c846b90 con 0x55866c835bb0 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40040/0 (socket says 172.21.15.38:40040) 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 -- 172.21.15.38:0/1364389340 learned_addr learned my addr 172.21.15.38:0/1364389340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:53.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9beffd640 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c842000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c845f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 -- 172.21.15.38:0/1364389340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 msgr2=0x55866c845f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c845f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 -- 172.21.15.38:0/1364389340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 msgr2=0x55866c842000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c842000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.835+0000 7ff9be7fc640 1 -- 172.21.15.38:0/1364389340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55866c826d90 con 0x55866c82b2f0 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40cc850 tx=0x7ff9b40d15b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70dac5de92ab621 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9a4ff9640 1 -- 172.21.15.38:0/1364389340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b401e040 con 0x55866c82b2f0 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9a4ff9640 1 -- 172.21.15.38:0/1364389340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b409f6e0 con 0x55866c82b2f0 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9a4ff9640 1 -- 172.21.15.38:0/1364389340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4010dc0 con 0x55866c82b2f0 2024-09-19T05:40:53.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/1364389340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 msgr2=0x55866c847ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40cc850 tx=0x7ff9b40d15b0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/1364389340 shutdown_connections 2024-09-19T05:40:53.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c845f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c847ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/1364389340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c842000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/1364389340 >> 172.21.15.38:0/1364389340 conn(0x55866c826f60 msgr2=0x55866c831c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/1364389340 shutdown_connections 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.836+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/1364389340 wait complete. 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 -- start start 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 Processor -- start 2024-09-19T05:40:53.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c84aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55866c847930 con 0x55866c82b2f0 2024-09-19T05:40:53.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55866c847260 con 0x55866c82a9d0 2024-09-19T05:40:53.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55866c848000 con 0x55866c835bb0 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9c7b0c740 10 client.0 _ll_register_callbacks cb 0x7ff9b801e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40056/0 (socket says 172.21.15.38:40056) 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 -- 172.21.15.38:0/2779091915 learned_addr learned my addr 172.21.15.38:0/2779091915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9bf7fe640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c84aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 -- 172.21.15.38:0/2779091915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 msgr2=0x55866c84aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c84aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 -- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 msgr2=0x55866c8503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:53.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.837+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9be7fc640 1 -- 172.21.15.38:0/2779091915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55866c84b3e0 con 0x55866c82b2f0 2024-09-19T05:40:53.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9be7fc640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4097790 tx=0x7ff9b40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7deced40a779dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:53.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:53.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40979a0 con 0x55866c82b2f0 2024-09-19T05:40:53.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff9b40d10f0 con 0x55866c82b2f0 2024-09-19T05:40:53.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b409f6e0 con 0x55866c82b2f0 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9b4097b40 con 0x55866c82b2f0 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9c7b0c740 10 client.15069 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55866c8afc90 con 0x55866c82b2f0 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b409f6e0 con 0x55866c82b2f0 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7ff9b40bb0b0 con 0x55866c82b2f0 2024-09-19T05:40:53.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.838+0000 7ff9c7b0c740 10 client.15069 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:40:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 20 client.15069 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c8b4230 0x55866c8b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 4 client.15069 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:40:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "status"}) -- 0x55866c798b10 con 0x55866c8b4230 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c8b4230 0x55866c8b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9beffd640 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c8b4230 0x55866c8b2f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f420360d0d10559 server_cookie=35b92d9749dc6eb3 in_seq=0 out_seq=0 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9a2ff5640 10 client.15069 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9a2ff5640 1 -- 172.21.15.38:0/2779091915 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55866c798b10 con 0x55866c8b4230 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9a2ff5640 10 client.15069 handle_command_reply: tid=0 2024-09-19T05:40:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 client.15069 shutdown 2024-09-19T05:40:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 msgr2=0x55866c84a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4097790 tx=0x7ff9b40047d0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 shutdown_connections 2024-09-19T05:40:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55866c8b4230 0x55866c8b2f40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55866c835bb0 0x55866c84aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.839+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55866c82b2f0 0x55866c84a760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c7b0c740 1 --2- 172.21.15.38:0/2779091915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55866c82a9d0 0x55866c8503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 >> 172.21.15.38:0/2779091915 conn(0x55866c826f60 msgr2=0x55866c8326d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9a2ff5640 0 client.15069 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 shutdown_connections 2024-09-19T05:40:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c7b0c740 1 -- 172.21.15.38:0/2779091915 wait complete. 2024-09-19T05:40:53.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c7b0c740 20 client.15069 trim_cache size 0 max 16384 2024-09-19T05:40:53.871 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:40:53.871 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:40:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-09-19T05:40:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24455, 2024-09-19T05:40:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:40:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:40:53.872 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 66.741605327000002, 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 89, 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 133.39967822599999 2024-09-19T05:40:53.873 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:40:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 msgr2=0x7ff9a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 0x7ff9a80a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005fd0 tx=0x7ff9b8009740 comp rx=0 tx=0).stop 2024-09-19T05:40:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 msgr2=0x7ff9c019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b4a0 tx=0x7ff9b400b960 comp rx=0 tx=0).stop 2024-09-19T05:40:53.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 shutdown_connections 2024-09-19T05:40:53.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff9a80a32d0 0x7ff9a80a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c010c180 0x7ff9c019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9c01036c0 0x7ff9c019ac60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 --2- 172.21.15.38:0/912325366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9c0102da0 0x7ff9c019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:53.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 >> 172.21.15.38:0/912325366 conn(0x7ff9c00fc7a0 msgr2=0x7ff9c00fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:53.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 shutdown_connections 2024-09-19T05:40:53.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:53.840+0000 7ff9c5733640 1 -- 172.21.15.38:0/912325366 wait complete. 2024-09-19T05:40:54.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T05:40:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:55.185 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:54 smithi038 ceph-mon[31227]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:55.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:54 smithi179 ceph-mon[35781]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:55.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:54 smithi047 ceph-mon[35327]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:40:55.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- 172.21.15.38:0/1075162701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe010c2f0 msgr2=0x7fcbe010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 --2- 172.21.15.38:0/1075162701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe010c2f0 0x7fcbe010e790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009900 tx=0x7fcbd802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:55.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- 172.21.15.38:0/1075162701 shutdown_connections 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 --2- 172.21.15.38:0/1075162701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe010c2f0 0x7fcbe010e790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 --2- 172.21.15.38:0/1075162701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe01037a0 0x7fcbe010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 --2- 172.21.15.38:0/1075162701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 0x7fcbe01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- 172.21.15.38:0/1075162701 >> 172.21.15.38:0/1075162701 conn(0x7fcbe00fc7d0 msgr2=0x7fcbe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- 172.21.15.38:0/1075162701 shutdown_connections 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- 172.21.15.38:0/1075162701 wait complete. 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 Processor -- start 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.417+0000 7fcbe7231640 1 -- start start 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 0x7fcbe0100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe010c2f0 0x7fcbe01014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01120f0 con 0x7fcbe01037a0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0111f70 con 0x7fcbe0102df0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0112270 con 0x7fcbe010c2f0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40074/0 (socket says 172.21.15.38:40074) 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 -- 172.21.15.38:0/3270000556 learned_addr learned my addr 172.21.15.38:0/3270000556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe010c2f0 0x7fcbe01014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 0x7fcbe0100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 -- 172.21.15.38:0/3270000556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe010c2f0 msgr2=0x7fcbe01014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe010c2f0 0x7fcbe01014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 -- 172.21.15.38:0/3270000556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 msgr2=0x7fcbe0100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 0x7fcbe0100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 -- 172.21.15.38:0/3270000556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe01a6a30 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbdffff640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c8c0 tx=0x7fcbd000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbddffb640 1 -- 172.21.15.38:0/3270000556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0007bb0 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbddffb640 1 -- 172.21.15.38:0/3270000556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0007d50 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.418+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe01a6d20 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.419+0000 7fcbddffb640 1 -- 172.21.15.38:0/3270000556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0018830 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.419+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a7250 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.421+0000 7fcbddffb640 1 -- 172.21.15.38:0/3270000556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcbd0018a10 con 0x7fcbe01037a0 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.421+0000 7fcbe960a740 1 Processor -- start 2024-09-19T05:40:55.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.421+0000 7fcbe960a740 1 -- start start 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.421+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e18fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe960a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2d72090 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2d19a70 con 0x55e3c2e16bb0 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe960a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2d6e050 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbddffb640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 0x7fcbcc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40088/0 (socket says 172.21.15.38:40088) 2024-09-19T05:40:55.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 -- 172.21.15.38:0/3754345737 learned_addr learned my addr 172.21.15.38:0/3754345737 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 0x7fcbcc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbddffb640 1 -- 172.21.15.38:0/3270000556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcbd00cfd70 con 0x7fcbe01037a0 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbdffff640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e18fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/3754345737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e16bb0 msgr2=0x55e3c2e18fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e18fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/3754345737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 msgr2=0x55e3c2e0bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/3754345737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c2d26130 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 0x7fcbcc0a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80096c0 tx=0x7fcbd8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.422+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e16670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009fd0 tx=0x7fcbc8003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27447ff9e74a97d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbd6ffd640 1 -- 172.21.15.38:0/3754345737 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8011440 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbd6ffd640 1 -- 172.21.15.38:0/3754345737 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8015030 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbd6ffd640 1 -- 172.21.15.38:0/3754345737 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80109c0 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 -- 172.21.15.38:0/3754345737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 msgr2=0x55e3c2e16670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e16670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009fd0 tx=0x7fcbc8003e00 comp rx=0 tx=0).stop 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 -- 172.21.15.38:0/3754345737 shutdown_connections 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e18fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e16670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 --2- 172.21.15.38:0/3754345737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.423+0000 7fcbe960a740 1 -- 172.21.15.38:0/3754345737 >> 172.21.15.38:0/3754345737 conn(0x55e3c2e07f60 msgr2=0x55e3c2e08340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 -- 172.21.15.38:0/3754345737 shutdown_connections 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 -- 172.21.15.38:0/3754345737 wait complete. 2024-09-19T05:40:55.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 -- start start 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 Processor -- start 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e19780 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.424+0000 7fcbe960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2d2cf20 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe960a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e19900 con 0x55e3c2e16bb0 2024-09-19T05:40:55.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe960a740 10 client.0 _ll_register_callbacks cb 0x55e3c2cf6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40104/0 (socket says 172.21.15.38:40104) 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/1348593592 learned_addr learned my addr 172.21.15.38:0/1348593592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbdffff640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 msgr2=0x55e3c2e13c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/1348593592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 msgr2=0x55e3c2e136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/1348593592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c2ae32b0 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009370 tx=0x7fcbc8005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefad0630d26fef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbbffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8029060 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcbc8010940 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.425+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc8015030 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcbc8030050 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3c2e24f90 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8015030 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fcbc8023070 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 10 client.15081 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 20 client.15081 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e27260 0x55e3c2e286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 4 client.15081 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c2d7f600 con 0x55e3c2e27260 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.426+0000 7fcbdffff640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e27260 0x55e3c2e286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.427+0000 7fcbbffff640 10 client.15081 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.427+0000 7fcbdffff640 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e27260 0x55e3c2e286d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b45489b762eb7bb5 server_cookie=1b5d97bfcdb57b39 in_seq=0 out_seq=0 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbbffff640 1 -- 172.21.15.38:0/1348593592 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3c2d7f600 con 0x55e3c2e27260 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbbffff640 10 client.15081 handle_command_reply: tid=0 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 client.15081 shutdown 2024-09-19T05:40:55.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 msgr2=0x55e3c2e16320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009370 tx=0x7fcbc8005ae0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 shutdown_connections 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e27260 0x55e3c2e286d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 --2- 172.21.15.38:0/1348593592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e16320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbbffff640 0 client.15081 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 >> 172.21.15.38:0/1348593592 conn(0x55e3c2e07f60 msgr2=0x55e3c2e0dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 shutdown_connections 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 1 -- 172.21.15.38:0/1348593592 wait complete. 2024-09-19T05:40:55.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.428+0000 7fcbe960a740 20 client.15081 trim_cache size 0 max 16384 2024-09-19T05:40:55.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 Processor -- start 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 -- start start 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e28000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e26a10 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e270e0 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe960a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e277b0 con 0x55e3c2e16bb0 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.446+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40106/0 (socket says 172.21.15.38:40106) 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/556707987 learned_addr learned my addr 172.21.15.38:0/556707987 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbdffff640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e28000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/556707987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 msgr2=0x55e3c2e13c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/556707987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 msgr2=0x55e3c2e28000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e28000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/556707987 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c2e0c120 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbdffff640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8003e40 tx=0x7fcbc8009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb6c12fafb0d0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbd67fc640 1 -- 172.21.15.38:0/556707987 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc803a070 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbd67fc640 1 -- 172.21.15.38:0/556707987 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8015030 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.447+0000 7fcbd67fc640 1 -- 172.21.15.38:0/556707987 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc801a700 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- 172.21.15.38:0/556707987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 msgr2=0x55e3c2e26b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8003e40 tx=0x7fcbc8009890 comp rx=0 tx=0).stop 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- 172.21.15.38:0/556707987 shutdown_connections 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e13c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 --2- 172.21.15.38:0/556707987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e26b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- 172.21.15.38:0/556707987 >> 172.21.15.38:0/556707987 conn(0x55e3c2e07f60 msgr2=0x55e3c2e0e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- 172.21.15.38:0/556707987 shutdown_connections 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- 172.21.15.38:0/556707987 wait complete. 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 -- start start 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.448+0000 7fcbe960a740 1 Processor -- start 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e28550 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e27e80 con 0x55e3c2e0c2f0 2024-09-19T05:40:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e3c2e28c20 con 0x55e3c2e16bb0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe960a740 10 client.0 _ll_register_callbacks cb 0x7fcbd8032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40118/0 (socket says 172.21.15.38:40118) 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/659726005 learned_addr learned my addr 172.21.15.38:0/659726005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbdffff640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe57ce640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 msgr2=0x55e3c2e103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/659726005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 msgr2=0x55e3c2e0fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e0fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 -- 172.21.15.38:0/659726005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c2e2f920 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbe4fcd640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8011690 tx=0x7fcbc8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d27fd60844c100 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.449+0000 7fcbbffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc803a040 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcbc8011d70 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc8015030 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcbc80117f0 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 10 client.15087 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3c2e30b30 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc8015030 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fcbc8023040 con 0x55e3c2e0b9d0 2024-09-19T05:40:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 10 client.15087 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 10 client.15087 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 20 client.15087 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e956d0 0x55e3c2e943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 4 client.15087 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.450+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "status"}) -- 0x55e3c2d79b10 con 0x55e3c2e956d0 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbdffff640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e956d0 0x55e3c2e943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbdffff640 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e956d0 0x55e3c2e943e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3366bcd46e55534 server_cookie=9393eebb5bef579f in_seq=0 out_seq=0 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbbffff640 10 client.15087 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbbffff640 1 -- 172.21.15.38:0/659726005 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e3c2d79b10 con 0x55e3c2e956d0 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbbffff640 10 client.15087 handle_command_reply: tid=0 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 client.15087 shutdown 2024-09-19T05:40:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 msgr2=0x55e3c2e0f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8011690 tx=0x7fcbc8005a10 comp rx=0 tx=0).stop 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 shutdown_connections 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3c2e16bb0 0x55e3c2e103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3c2e0c2f0 0x55e3c2e0fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55e3c2e956d0 0x55e3c2e943e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 --2- 172.21.15.38:0/659726005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e3c2e0b9d0 0x55e3c2e0f930 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbbffff640 0 client.15087 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.451+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 >> 172.21.15.38:0/659726005 conn(0x55e3c2e07f60 msgr2=0x55e3c2e0d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 shutdown_connections 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe960a740 1 -- 172.21.15.38:0/659726005 wait complete. 2024-09-19T05:40:55.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe960a740 20 client.15087 trim_cache size 0 max 16384 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24475, 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 69.360643651000004, 2024-09-19T05:40:55.462 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:40:55.463 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 89, 2024-09-19T05:40:55.463 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:40:55.463 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 140.23745813100001 2024-09-19T05:40:55.463 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 msgr2=0x7fcbcc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 0x7fcbcc0a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80096c0 tx=0x7fcbd8009260 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 msgr2=0x7fcbe0100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c8c0 tx=0x7fcbd000cd80 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 shutdown_connections 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcbcc0a3350 0x7fcbcc0a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe010c2f0 0x7fcbe01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbe01037a0 0x7fcbe0100f90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 --2- 172.21.15.38:0/3270000556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe0102df0 0x7fcbe0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.452+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 >> 172.21.15.38:0/3270000556 conn(0x7fcbe00fc7d0 msgr2=0x7fcbe00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:55.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.453+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 shutdown_connections 2024-09-19T05:40:55.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:55.453+0000 7fcbe7231640 1 -- 172.21.15.38:0/3270000556 wait complete. 2024-09-19T05:40:56.206 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:40:56.464 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:57.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 -- 172.21.15.38:0/555800320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 msgr2=0x7f835c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 --2- 172.21.15.38:0/555800320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c1021c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8358009900 tx=0x7f835802a410 comp rx=0 tx=0).stop 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 -- 172.21.15.38:0/555800320 shutdown_connections 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 --2- 172.21.15.38:0/555800320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c106f10 0x7f835c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 --2- 172.21.15.38:0/555800320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c102790 0x7f835c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 --2- 172.21.15.38:0/555800320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c1021c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 -- 172.21.15.38:0/555800320 >> 172.21.15.38:0/555800320 conn(0x7f835c0fd8c0 msgr2=0x7f835c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 -- 172.21.15.38:0/555800320 shutdown_connections 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.994+0000 7f836451e640 1 -- 172.21.15.38:0/555800320 wait complete. 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 Processor -- start 2024-09-19T05:40:57.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 -- start start 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c102790 0x7f835c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c106f10 0x7f835c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f835c111f70 con 0x7f835c101de0 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f835c111df0 con 0x7f835c106f10 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f836451e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f835c1120f0 con 0x7f835c102790 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40138/0 (socket says 172.21.15.38:40138) 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 -- 172.21.15.38:0/2869189846 learned_addr learned my addr 172.21.15.38:0/2869189846 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f8361ab9640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c102790 0x7f835c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f8362abb640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c106f10 0x7f835c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 -- 172.21.15.38:0/2869189846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c102790 msgr2=0x7f835c19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c102790 0x7f835c19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 -- 172.21.15.38:0/2869189846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c106f10 msgr2=0x7f835c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c106f10 0x7f835c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 -- 172.21.15.38:0/2869189846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c19f680 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.995+0000 7f83622ba640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8358006010 tx=0x7f83580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.996+0000 7f83537fe640 1 -- 172.21.15.38:0/2869189846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835803f030 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.996+0000 7f83537fe640 1 -- 172.21.15.38:0/2869189846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358003ce0 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.996+0000 7f83537fe640 1 -- 172.21.15.38:0/2869189846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358035070 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.996+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c19f910 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.996+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c19fda0 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.998+0000 7f83537fe640 1 -- 172.21.15.38:0/2869189846 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8358042030 con 0x7f835c101de0 2024-09-19T05:40:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.998+0000 7f83668f7740 1 Processor -- start 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 -- start start 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf58db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf63670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561a7cebf090 con 0x561a7cf63bb0 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561a7ce66a70 con 0x561a7cf592f0 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83668f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a7cebb050 con 0x561a7cf589d0 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83537fe640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 0x7f83480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40144/0 (socket says 172.21.15.38:40144) 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 -- 172.21.15.38:0/3232641303 learned_addr learned my addr 172.21.15.38:0/3232641303 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8362abb640 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf58db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83537fe640 1 -- 172.21.15.38:0/2869189846 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83580f03d0 con 0x7f835c101de0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8362abb640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 0x7f83480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f83622ba640 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf63670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 -- 172.21.15.38:0/3232641303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 msgr2=0x561a7cf58db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf58db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 -- 172.21.15.38:0/3232641303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 msgr2=0x561a7cf63670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf63670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:56.999+0000 7f8361ab9640 1 -- 172.21.15.38:0/3232641303 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7ce73130 con 0x561a7cf63bb0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f8361ab9640 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f834c00bc80 tx=0x7f834c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dec3d2b09c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f8362abb640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 0x7f83480a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8354005c40 tx=0x7f8354004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f8350ff9640 1 -- 172.21.15.38:0/3232641303 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c018be0 con 0x561a7cf63bb0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f8350ff9640 1 -- 172.21.15.38:0/3232641303 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c018d80 con 0x561a7cf63bb0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f8350ff9640 1 -- 172.21.15.38:0/3232641303 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c01f670 con 0x561a7cf63bb0 2024-09-19T05:40:57.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f83668f7740 1 -- 172.21.15.38:0/3232641303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 msgr2=0x561a7cf65fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.000+0000 7f83668f7740 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f834c00bc80 tx=0x7f834c00f730 comp rx=0 tx=0).stop 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- 172.21.15.38:0/3232641303 shutdown_connections 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cf65fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf63670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- 172.21.15.38:0/3232641303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf58db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- 172.21.15.38:0/3232641303 >> 172.21.15.38:0/3232641303 conn(0x561a7cf54f60 msgr2=0x561a7cf55340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- 172.21.15.38:0/3232641303 shutdown_connections 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- 172.21.15.38:0/3232641303 wait complete. 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- start start 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 Processor -- start 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf60ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf66780 con 0x561a7cf589d0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561a7ce79f20 con 0x561a7cf63bb0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf66900 con 0x561a7cf592f0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83668f7740 10 client.0 _ll_register_callbacks cb 0x561a7ce43030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40160/0 (socket says 172.21.15.38:40160) 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.001+0000 7f83622ba640 1 -- 172.21.15.38:0/2062029264 learned_addr learned my addr 172.21.15.38:0/2062029264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 -- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 msgr2=0x561a7cf607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8361ab9640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8362abb640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf60ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 -- 172.21.15.38:0/2062029264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 msgr2=0x561a7cf60ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf60ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 -- 172.21.15.38:0/2062029264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7cc302b0 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8361ab9640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8362abb640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf60ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83622ba640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f83580b89f0 tx=0x7f83580bc420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ef3038f0d2ce55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8345ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358049070 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83580edc10 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834c018d80 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83580044e0 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83668f7740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a7cf720a0 con 0x561a7cf589d0 2024-09-19T05:40:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.002+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c018d80 con 0x561a7cf589d0 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f83580bc7f0 con 0x561a7cf589d0 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 20 client.15099 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cf74750 0x561a7cf75bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 4 client.15099 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7cecc600 con 0x561a7cf74750 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f8361ab9640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cf74750 0x561a7cf75bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f8345ffb640 10 client.15099 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.003+0000 7f8361ab9640 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cf74750 0x561a7cf75bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e18977af89ff20b6 server_cookie=546b5d7b57e09564 in_seq=0 out_seq=0 2024-09-19T05:40:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.004+0000 7f8345ffb640 1 -- 172.21.15.38:0/2062029264 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a7cecc600 con 0x561a7cf74750 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.004+0000 7f8345ffb640 10 client.15099 handle_command_reply: tid=0 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.004+0000 7f83668f7740 1 client.15099 shutdown 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.004+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 msgr2=0x561a7cf63450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.004+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f83580b89f0 tx=0x7f83580bc420 comp rx=0 tx=0).stop 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 shutdown_connections 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cf74750 0x561a7cf75bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf60ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f8345ffb640 0 client.15099 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 --2- 172.21.15.38:0/2062029264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf63450 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 >> 172.21.15.38:0/2062029264 conn(0x561a7cf54f60 msgr2=0x561a7cf5ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 shutdown_connections 2024-09-19T05:40:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 1 -- 172.21.15.38:0/2062029264 wait complete. 2024-09-19T05:40:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.005+0000 7f83668f7740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T05:40:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 Processor -- start 2024-09-19T05:40:57.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 -- start start 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf5da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf73f00 con 0x561a7cf589d0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf745d0 con 0x561a7cf63bb0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83668f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf74ca0 con 0x561a7cf592f0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83622ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40162/0 (socket says 172.21.15.38:40162) 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f83622ba640 1 -- 172.21.15.38:0/3682844058 learned_addr learned my addr 172.21.15.38:0/3682844058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f8362abb640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf5da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.023+0000 7f8361ab9640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 -- 172.21.15.38:0/3682844058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 msgr2=0x561a7cf60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 -- 172.21.15.38:0/3682844058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 msgr2=0x561a7cf5da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf5da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 -- 172.21.15.38:0/3682844058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7cf66890 con 0x561a7cf589d0 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f8362abb640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf5da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f83622ba640 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8358046040 tx=0x7f83580bc490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c173ce482e0e604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f8347fff640 1 -- 172.21.15.38:0/3682844058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358049040 con 0x561a7cf589d0 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f8347fff640 1 -- 172.21.15.38:0/3682844058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83580bcc90 con 0x561a7cf589d0 2024-09-19T05:40:57.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.024+0000 7f8347fff640 1 -- 172.21.15.38:0/3682844058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580edc10 con 0x561a7cf589d0 2024-09-19T05:40:57.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- 172.21.15.38:0/3682844058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 msgr2=0x561a7cf75bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8358046040 tx=0x7f83580bc490 comp rx=0 tx=0).stop 2024-09-19T05:40:57.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- 172.21.15.38:0/3682844058 shutdown_connections 2024-09-19T05:40:57.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf63bb0 0x561a7cf5da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf592f0 0x561a7cf60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- 172.21.15.38:0/3682844058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf589d0 0x561a7cf75bc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- 172.21.15.38:0/3682844058 >> 172.21.15.38:0/3682844058 conn(0x561a7cf54f60 msgr2=0x561a7cf55340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- 172.21.15.38:0/3682844058 shutdown_connections 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- 172.21.15.38:0/3682844058 wait complete. 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- start start 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 Processor -- start 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf79540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf75a40 con 0x561a7cf63bb0 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf75370 con 0x561a7cf592f0 2024-09-19T05:40:57.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a7cf76110 con 0x561a7cf589d0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.025+0000 7f83668f7740 10 client.0 _ll_register_callbacks cb 0x7f835803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40174/0 (socket says 172.21.15.38:40174) 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 -- 172.21.15.38:0/3709767467 learned_addr learned my addr 172.21.15.38:0/3709767467 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f83622ba640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8361ab9640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf79540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 -- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 msgr2=0x561a7cf79000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf79000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 -- 172.21.15.38:0/3709767467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 msgr2=0x561a7cf79540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf79540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 -- 172.21.15.38:0/3709767467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7cfdd390 con 0x561a7cf63bb0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8362abb640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8354015010 tx=0x7f8354004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f968b828aea7a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:57.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8345ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354024070 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8354004b20 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.026+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83580bcc90 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8354031050 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a7cfde5e0 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83580bcc90 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f8354005160 con 0x561a7cf63bb0 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 20 client.15105 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cfe3200 0x561a7cfe1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 4 client.15105 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:40:57.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "status"}) -- 0x561a7cec6b10 con 0x561a7cfe3200 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.027+0000 7f83622ba640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cfe3200 0x561a7cfe1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83622ba640 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cfe3200 0x561a7cfe1f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb872b4ce03ebf66 server_cookie=5652bafc499aa1a2 in_seq=0 out_seq=0 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f8345ffb640 10 client.15105 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f8345ffb640 1 -- 172.21.15.38:0/3709767467 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561a7cec6b10 con 0x561a7cfe3200 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f8345ffb640 10 client.15105 handle_command_reply: tid=0 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 client.15105 shutdown 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 msgr2=0x561a7cfdce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8354015010 tx=0x7f8354004ab0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 shutdown_connections 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x561a7cfe3200 0x561a7cfe1f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561a7cf63bb0 0x561a7cfdce50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a7cf592f0 0x561a7cf79540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f8345ffb640 0 client.15105 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 --2- 172.21.15.38:0/3709767467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a7cf589d0 0x561a7cf79000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 >> 172.21.15.38:0/3709767467 conn(0x561a7cf54f60 msgr2=0x561a7cf5f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 shutdown_connections 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 1 -- 172.21.15.38:0/3709767467 wait complete. 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.028+0000 7f83668f7740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-19T05:40:57.036 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24481, 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 71.944862628999999, 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 89, 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 139.199709364 2024-09-19T05:40:57.037 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 msgr2=0x7f83480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 0x7f83480a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8354005c40 tx=0x7f8354004a40 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 msgr2=0x7f835c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8358006010 tx=0x7f83580042f0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 shutdown_connections 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83480a3350 0x7f83480a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f835c106f10 0x7f835c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f835c102790 0x7f835c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 --2- 172.21.15.38:0/2869189846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f835c101de0 0x7f835c19a6d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 >> 172.21.15.38:0/2869189846 conn(0x7f835c0fd8c0 msgr2=0x7f835c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 shutdown_connections 2024-09-19T05:40:57.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:57.029+0000 7f836451e640 1 -- 172.21.15.38:0/2869189846 wait complete. 2024-09-19T05:40:57.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:56 smithi179 ceph-mon[35781]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:57.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:56 smithi047 ceph-mon[35327]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:57.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:56 smithi038 ceph-mon[31227]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:40:57.999 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:40:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 -- 172.21.15.38:0/191684825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c100e20 msgr2=0x7f609c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 --2- 172.21.15.38:0/191684825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c100e20 0x7f609c101200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6098009900 tx=0x7f609802a480 comp rx=0 tx=0).stop 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 -- 172.21.15.38:0/191684825 shutdown_connections 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 --2- 172.21.15.38:0/191684825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c102ef0 0x7f609c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 --2- 172.21.15.38:0/191684825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 --2- 172.21.15.38:0/191684825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c100e20 0x7f609c101200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 -- 172.21.15.38:0/191684825 >> 172.21.15.38:0/191684825 conn(0x7f609c0fc7d0 msgr2=0x7f609c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 -- 172.21.15.38:0/191684825 shutdown_connections 2024-09-19T05:40:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.739+0000 7f60a2c37640 1 -- 172.21.15.38:0/191684825 wait complete. 2024-09-19T05:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 Processor -- start 2024-09-19T05:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 -- start start 2024-09-19T05:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c100e20 0x7f609c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f609c110fa0 con 0x7f609c102ef0 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f609c110e20 con 0x7f609c102570 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a2c37640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111120 con 0x7f609c100e20 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53608/0 (socket says 172.21.15.38:53608) 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 -- 172.21.15.38:0/3050267806 learned_addr learned my addr 172.21.15.38:0/3050267806 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a09d3640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c100e20 0x7f609c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f6093fff640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 -- 172.21.15.38:0/3050267806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c100e20 msgr2=0x7f609c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c100e20 0x7f609c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 -- 172.21.15.38:0/3050267806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 msgr2=0x7f609c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.740+0000 7f60a11d4640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f60a11d4640 1 -- 172.21.15.38:0/3050267806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c19c450 con 0x7f609c102ef0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f6093fff640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f60a11d4640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6094007c30 tx=0x7f609400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f6091ffb640 1 -- 172.21.15.38:0/3050267806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094019020 con 0x7f609c102ef0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f6091ffb640 1 -- 172.21.15.38:0/3050267806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094007e40 con 0x7f609c102ef0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c19c740 con 0x7f609c102ef0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f6091ffb640 1 -- 172.21.15.38:0/3050267806 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094005590 con 0x7f609c102ef0 2024-09-19T05:40:58.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.741+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c069f70 con 0x7f609c102ef0 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.743+0000 7f6091ffb640 1 -- 172.21.15.38:0/3050267806 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f60940057b0 con 0x7f609c102ef0 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 Processor -- start 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 -- start start 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9f02090 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9ea9a70 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a5010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9efe050 con 0x55d5a9f9c2f0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53624/0 (socket says 172.21.15.38:53624) 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a09d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 -- 172.21.15.38:0/1723065233 learned_addr learned my addr 172.21.15.38:0/1723065233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6091ffb640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 0x7f60840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6091ffb640 1 -- 172.21.15.38:0/3050267806 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f60940d1750 con 0x7f609c102ef0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a11d4640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 0x7f60840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 -- 172.21.15.38:0/1723065233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 msgr2=0x55d5a9fa6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 -- 172.21.15.38:0/1723065233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9fa6bb0 msgr2=0x55d5a9fa8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f6093fff640 1 -- 172.21.15.38:0/1723065233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a9eb6130 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.744+0000 7f60a09d3640 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.745+0000 7f6093fff640 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f608c005fc0 tx=0x7f608c00c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf6ae723a95818f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.745+0000 7f6082ffd640 1 -- 172.21.15.38:0/1723065233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c005020 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.745+0000 7f60a11d4640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 0x7f60840a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f609400ef30 tx=0x7f609400ee80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.745+0000 7f6082ffd640 1 -- 172.21.15.38:0/1723065233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c0047a0 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.745+0000 7f6082ffd640 1 -- 172.21.15.38:0/1723065233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c010910 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- 172.21.15.38:0/1723065233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 msgr2=0x55d5a9f9bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f608c005fc0 tx=0x7f608c00c640 comp rx=0 tx=0).stop 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- 172.21.15.38:0/1723065233 shutdown_connections 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 --2- 172.21.15.38:0/1723065233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9f9bdb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- 172.21.15.38:0/1723065233 >> 172.21.15.38:0/1723065233 conn(0x55d5a9f97f60 msgr2=0x55d5a9f98340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- 172.21.15.38:0/1723065233 shutdown_connections 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- 172.21.15.38:0/1723065233 wait complete. 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 -- start start 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.746+0000 7f60a5010740 1 Processor -- start 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fa6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fa9780 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9ebcf20 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fa9900 con 0x55d5a9f9c2f0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a5010740 10 client.0 _ll_register_callbacks cb 0x55d5a9e86030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53630/0 (socket says 172.21.15.38:53630) 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 -- 172.21.15.38:0/2217951952 learned_addr learned my addr 172.21.15.38:0/2217951952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f60a09d3640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fa6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 -- 172.21.15.38:0/2217951952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 msgr2=0x55d5a9fa37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 -- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 msgr2=0x55d5a9fa6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fa6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 -- 172.21.15.38:0/2217951952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a9c732b0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f6093fff640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f608c00c470 tx=0x7f608c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ec018c1b036e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.747+0000 7f607bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c005210 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f608c01f030 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c0047a0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f608c0053b0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5a9fb50a0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c0047a0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f608c010a20 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:58.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 20 client.15117 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5a9fb7320 0x55d5a9fb8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 4 client.15117 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.748+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a9f0f600 con 0x55d5a9fb7320 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.749+0000 7f60a09d3640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5a9fb7320 0x55d5a9fb8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.749+0000 7f60a09d3640 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5a9fb7320 0x55d5a9fb8790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cabeb747b2edf19f server_cookie=11869aecf231ab7 in_seq=0 out_seq=0 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.749+0000 7f607bfff640 10 client.15117 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f607bfff640 1 -- 172.21.15.38:0/2217951952 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5a9f0f600 con 0x55d5a9fb7320 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f607bfff640 10 client.15117 handle_command_reply: tid=0 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 client.15117 shutdown 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 msgr2=0x55d5a9fa3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f608c00c470 tx=0x7f608c014040 comp rx=0 tx=0).stop 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 shutdown_connections 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5a9fb7320 0x55d5a9fb8790 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa3ce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 --2- 172.21.15.38:0/2217951952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fa6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 >> 172.21.15.38:0/2217951952 conn(0x55d5a9f97f60 msgr2=0x55d5a9f9dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f607bfff640 0 client.15117 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 shutdown_connections 2024-09-19T05:40:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 1 -- 172.21.15.38:0/2217951952 wait complete. 2024-09-19T05:40:58.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.750+0000 7f60a5010740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 Processor -- start 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 -- start start 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb6ad0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb71a0 con 0x55d5a9f9c2f0 2024-09-19T05:40:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a5010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb7870 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53642/0 (socket says 172.21.15.38:53642) 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 -- 172.21.15.38:0/2083022722 learned_addr learned my addr 172.21.15.38:0/2083022722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f60a09d3640 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 -- 172.21.15.38:0/2083022722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9b9d0 msgr2=0x55d5a9fb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fb79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 -- 172.21.15.38:0/2083022722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9c2f0 msgr2=0x55d5a9fa2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.769+0000 7f6093fff640 1 -- 172.21.15.38:0/2083022722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a9fa42e0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f6093fff640 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f608c004fd0 tx=0x7f608c00c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa7a2e05ce410 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60827fc640 1 -- 172.21.15.38:0/2083022722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c005210 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60827fc640 1 -- 172.21.15.38:0/2083022722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c004bd0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60827fc640 1 -- 172.21.15.38:0/2083022722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c0035f0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 -- 172.21.15.38:0/2083022722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 msgr2=0x55d5a9fa25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f608c004fd0 tx=0x7f608c00c4a0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 -- 172.21.15.38:0/2083022722 shutdown_connections 2024-09-19T05:40:58.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fa25b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fa2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 --2- 172.21.15.38:0/2083022722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.770+0000 7f60a5010740 1 -- 172.21.15.38:0/2083022722 >> 172.21.15.38:0/2083022722 conn(0x55d5a9f97f60 msgr2=0x55d5a9f98340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- 172.21.15.38:0/2083022722 shutdown_connections 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- 172.21.15.38:0/2083022722 wait complete. 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- start start 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 Processor -- start 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fbbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb8610 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb7f40 con 0x55d5a9f9b9d0 2024-09-19T05:40:58.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5a9fb8ce0 con 0x55d5a9f9c2f0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f60a5010740 10 client.0 _ll_register_callbacks cb 0x7f6098009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.771+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53658/0 (socket says 172.21.15.38:53658) 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 -- 172.21.15.38:0/3734569837 learned_addr learned my addr 172.21.15.38:0/3734569837 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f60a09d3640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fbbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 -- 172.21.15.38:0/3734569837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 msgr2=0x55d5a9fbbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fbbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 -- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 msgr2=0x55d5a9fbb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fbb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 -- 172.21.15.38:0/3734569837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5aa01fa20 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f6093fff640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f608c004ec0 tx=0x7f608c004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b4945a561f9ade server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f607bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:40:58.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c005210 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f608c01f040 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c004bd0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f608c00cde0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f60a5010740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.772+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5aa020c80 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c004bd0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f608c01a9c0 con 0x55d5a9fa6bb0 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 20 client.15123 populate_metadata read hostname 'smithi038' 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5aa025450 0x55d5aa024160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:40:58.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 4 client.15123 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "status"}) -- 0x55d5a9f09b10 con 0x55d5aa025450 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.773+0000 7f60a09d3640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5aa025450 0x55d5aa024160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a09d3640 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5aa025450 0x55d5aa024160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4de7d897a8db78cd server_cookie=41a00af9dde22288 in_seq=0 out_seq=0 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f607bfff640 10 client.15123 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f607bfff640 1 -- 172.21.15.38:0/3734569837 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55d5a9f09b10 con 0x55d5aa025450 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f607bfff640 10 client.15123 handle_command_reply: tid=0 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 client.15123 shutdown 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 msgr2=0x55d5a9fbc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f608c004ec0 tx=0x7f608c004660 comp rx=0 tx=0).stop 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 shutdown_connections 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55d5aa025450 0x55d5aa024160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d5a9fa6bb0 0x55d5a9fbc140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5a9f9c2f0 0x55d5a9fbbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f607bfff640 0 client.15123 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 --2- 172.21.15.38:0/3734569837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5a9f9b9d0 0x55d5a9fbb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 >> 172.21.15.38:0/3734569837 conn(0x55d5a9f97f60 msgr2=0x55d5a9fa1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 shutdown_connections 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 1 -- 172.21.15.38:0/3734569837 wait complete. 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.774+0000 7f60a5010740 20 client.15123 trim_cache size 0 max 16384 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-19T05:40:58.781 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24485, 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 74.701675158, 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 89, 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 126.919487003 2024-09-19T05:40:58.782 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 msgr2=0x7f60840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 0x7f60840a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f609400ef30 tx=0x7f609400ee80 comp rx=0 tx=0).stop 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 msgr2=0x7f609c19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6094007c30 tx=0x7f609400e420 comp rx=0 tx=0).stop 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 shutdown_connections 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60840a3350 0x7f60840a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f609c102ef0 0x7f609c19bd70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f609c102570 0x7f609c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 --2- 172.21.15.38:0/3050267806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f609c100e20 0x7f609c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:40:58.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 >> 172.21.15.38:0/3050267806 conn(0x7f609c0fc7d0 msgr2=0x7f609c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:40:58.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 shutdown_connections 2024-09-19T05:40:58.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:40:58.775+0000 7f60a2c37640 1 -- 172.21.15.38:0/3050267806 wait complete. 2024-09-19T05:40:59.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:58 smithi179 ceph-mon[35781]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:59.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:58 smithi047 ceph-mon[35327]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:59.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:58 smithi038 ceph-mon[31227]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:40:59.612 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:40:59.897 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:00.197 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:40:59 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:41:00.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:40:59 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:41:00.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:40:59 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:41:00.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 -- 172.21.15.38:0/2766096197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074f40 msgr2=0x7f6fe410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 --2- 172.21.15.38:0/2766096197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074f40 0x7f6fe410de80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc02a480 comp rx=0 tx=0).stop 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 -- 172.21.15.38:0/2766096197 shutdown_connections 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 --2- 172.21.15.38:0/2766096197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 --2- 172.21.15.38:0/2766096197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074f40 0x7f6fe410de80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 --2- 172.21.15.38:0/2766096197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074620 0x7f6fe4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 -- 172.21.15.38:0/2766096197 >> 172.21.15.38:0/2766096197 conn(0x7f6fe40fd8c0 msgr2=0x7f6fe40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 -- 172.21.15.38:0/2766096197 shutdown_connections 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.384+0000 7f6fe8842640 1 -- 172.21.15.38:0/2766096197 wait complete. 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 Processor -- start 2024-09-19T05:41:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 -- start start 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074f40 0x7f6fe419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe41141b0 con 0x7f6fe4074620 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4114030 con 0x7f6fe410e3c0 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe8842640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe4114330 con 0x7f6fe4074f40 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53664/0 (socket says 172.21.15.38:53664) 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 -- 172.21.15.38:0/1289752665 learned_addr learned my addr 172.21.15.38:0/1289752665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074f40 0x7f6fe419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 -- 172.21.15.38:0/1289752665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074f40 msgr2=0x7f6fe419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074f40 0x7f6fe419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 -- 172.21.15.38:0/1289752665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 msgr2=0x7f6fe419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe259c640 1 -- 172.21.15.38:0/1289752665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419f660 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.385+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e9f0 tx=0x7f6fdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1289752665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00cd80 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1289752665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc005500 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe419f950 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1289752665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc010690 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.386+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a7230 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1289752665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6fdc0046c0 con 0x7f6fe4074620 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 Processor -- start 2024-09-19T05:41:00.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 -- start start 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925446670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925448fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b9253a2090 con 0x55b92543b9d0 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b925349a70 con 0x55b92543c2f0 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.389+0000 7f6feac1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b92539e050 con 0x55b925446bb0 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53674/0 (socket says 172.21.15.38:53674) 2024-09-19T05:41:00.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 -- 172.21.15.38:0/3624930409 learned_addr learned my addr 172.21.15.38:0/3624930409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925448fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe259c640 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925446670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 -- 172.21.15.38:0/3624930409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 msgr2=0x55b925448fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925448fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 -- 172.21.15.38:0/3624930409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 msgr2=0x55b925446670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925446670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fdb7fe640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 0x7f6fd00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 -- 172.21.15.38:0/3624930409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b925356130 con 0x55b92543b9d0 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 0x7f6fd00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1289752665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6fdc0d0230 con 0x7f6fe4074620 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009f10 tx=0x7f6fd400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc52794e23ed0a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fd8ff9640 1 -- 172.21.15.38:0/3624930409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4003b70 con 0x55b92543b9d0 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fd8ff9640 1 -- 172.21.15.38:0/3624930409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4003d10 con 0x55b92543b9d0 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.390+0000 7f6fd8ff9640 1 -- 172.21.15.38:0/3624930409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40119a0 con 0x55b92543b9d0 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 -- 172.21.15.38:0/3624930409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 msgr2=0x55b92543bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 0x7f6fd00a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000c00 tx=0x7f6fcc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009f10 tx=0x7f6fd400f730 comp rx=0 tx=0).stop 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 -- 172.21.15.38:0/3624930409 shutdown_connections 2024-09-19T05:41:00.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925448fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925446670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 --2- 172.21.15.38:0/3624930409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92543bdb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 -- 172.21.15.38:0/3624930409 >> 172.21.15.38:0/3624930409 conn(0x55b925437f60 msgr2=0x55b925438340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 -- 172.21.15.38:0/3624930409 shutdown_connections 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.391+0000 7f6feac1b740 1 -- 172.21.15.38:0/3624930409 wait complete. 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 -- start start 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 Processor -- start 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 0x55b925446420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 0x55b925443cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b925449780 con 0x55b92543c2f0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b92535cf20 con 0x55b925446bb0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b925449900 con 0x55b92543b9d0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6feac1b740 10 client.0 _ll_register_callbacks cb 0x55b925326030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6fe1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6fe1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53680/0 (socket says 172.21.15.38:53680) 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6fe1d9b640 1 -- 172.21.15.38:0/152396943 learned_addr learned my addr 172.21.15.38:0/152396943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6fe259c640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 0x55b925446420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.392+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 0x55b925443cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 -- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 msgr2=0x55b925446420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 0x55b925446420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 -- 172.21.15.38:0/152396943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 msgr2=0x55b925443cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 0x55b925443cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 -- 172.21.15.38:0/152396943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9251132b0 con 0x55b92543c2f0 2024-09-19T05:41:00.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe259c640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 0x55b925446420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 0x55b925443cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e78ddbea8a5fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc04e070 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6fcc004b90 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd4003d10 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc05a050 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6feac1b740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b925455070 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd4003d10 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.393+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6fcc0036c0 con 0x55b92543c2f0 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:00.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 20 client.15135 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b925457690 0x55b925458b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 4 client.15135 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9253af600 con 0x55b925457690 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6fe259c640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b925457690 0x55b925458b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6fe259c640 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b925457690 0x55b925458b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55acb222234892a server_cookie=d7eca74c0407adf1 in_seq=0 out_seq=0 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.394+0000 7f6fc9ffb640 10 client.15135 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/152396943 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9253af600 con 0x55b925457690 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6fc9ffb640 10 client.15135 handle_command_reply: tid=0 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 client.15135 shutdown 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 msgr2=0x55b925443770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc009260 comp rx=0 tx=0).stop 2024-09-19T05:41:00.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 shutdown_connections 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b925457690 0x55b925458b00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b925446bb0 0x55b925443cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543c2f0 0x55b925443770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6fc9ffb640 0 client.15135 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 --2- 172.21.15.38:0/152396943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b92543b9d0 0x55b925446420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 >> 172.21.15.38:0/152396943 conn(0x55b925437f60 msgr2=0x55b92543dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 shutdown_connections 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 1 -- 172.21.15.38:0/152396943 wait complete. 2024-09-19T05:41:00.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.395+0000 7f6feac1b740 20 client.15135 trim_cache size 0 max 16384 2024-09-19T05:41:00.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 Processor -- start 2024-09-19T05:41:00.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 -- start start 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925457d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925460de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b925456e40 con 0x55b92543b9d0 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b925457510 con 0x55b92543c2f0 2024-09-19T05:41:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6feac1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b925457be0 con 0x55b925446bb0 2024-09-19T05:41:00.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53696/0 (socket says 172.21.15.38:53696) 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6fe259c640 1 -- 172.21.15.38:0/1419486473 learned_addr learned my addr 172.21.15.38:0/1419486473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925460de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.414+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925457d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 -- 172.21.15.38:0/1419486473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 msgr2=0x55b925460de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925460de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 -- 172.21.15.38:0/1419486473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 msgr2=0x55b925457d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925457d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 -- 172.21.15.38:0/1419486473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b925449890 con 0x55b92543b9d0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fe259c640 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0cde00 tx=0x7f6fdc0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58f35cec3f248db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fcbfff640 1 -- 172.21.15.38:0/1419486473 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc09c160 con 0x55b92543b9d0 2024-09-19T05:41:00.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fcbfff640 1 -- 172.21.15.38:0/1419486473 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc09ca90 con 0x55b92543b9d0 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.415+0000 7f6fcbfff640 1 -- 172.21.15.38:0/1419486473 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc09bcf0 con 0x55b92543b9d0 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- 172.21.15.38:0/1419486473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 msgr2=0x55b925456fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0cde00 tx=0x7f6fdc0bc040 comp rx=0 tx=0).stop 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- 172.21.15.38:0/1419486473 shutdown_connections 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b925460de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b925457d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 --2- 172.21.15.38:0/1419486473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b925456fc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- 172.21.15.38:0/1419486473 >> 172.21.15.38:0/1419486473 conn(0x55b925437f60 msgr2=0x55b925442040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- 172.21.15.38:0/1419486473 shutdown_connections 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- 172.21.15.38:0/1419486473 wait complete. 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.416+0000 7f6feac1b740 1 -- start start 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 Processor -- start 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b92545c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b9254bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55b925458980 con 0x55b92543b9d0 2024-09-19T05:41:00.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b9254582b0 con 0x55b92543c2f0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b925459050 con 0x55b925446bb0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6feac1b740 10 client.0 _ll_register_callbacks cb 0x7f6fcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53700/0 (socket says 172.21.15.38:53700) 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 -- 172.21.15.38:0/963310714 learned_addr learned my addr 172.21.15.38:0/963310714 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe2d9d640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b9254bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b92545c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 -- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 msgr2=0x55b9254bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b9254bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 -- 172.21.15.38:0/963310714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 msgr2=0x55b92545c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:00.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b92545c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.417+0000 7f6fe259c640 1 -- 172.21.15.38:0/963310714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9254c02d0 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fe259c640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc098d10 tx=0x7f6fdc0d3bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42787d36dee86008 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc09bed0 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6fdc0a4d70 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fdc09ca90 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc09bcf0 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6feac1b740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9254c1450 con 0x55b92543b9d0 2024-09-19T05:41:00.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc09ca90 con 0x55b92543b9d0 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6fdc09b7e0 con 0x55b92543b9d0 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.418+0000 7f6feac1b740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 20 client.15141 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b9254c5b00 0x55b9254c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 4 client.15141 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:00.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "status"}) -- 0x55b9253a9b10 con 0x55b9254c5b00 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b9254c5b00 0x55b9254c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6fe1d9b640 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b9254c5b00 0x55b9254c4810 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf97a81d3d163c2 server_cookie=e612950b03a77450 in_seq=0 out_seq=0 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6fc9ffb640 10 client.15141 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6fc9ffb640 1 -- 172.21.15.38:0/963310714 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b9253a9b10 con 0x55b9254c5b00 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6fc9ffb640 10 client.15141 handle_command_reply: tid=0 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.419+0000 7f6feac1b740 1 client.15141 shutdown 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 msgr2=0x55b92545be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc098d10 tx=0x7f6fdc0d3bd0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 shutdown_connections 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55b9254c5b00 0x55b9254c4810 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b925446bb0 0x55b9254bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b92543c2f0 0x55b92545c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 --2- 172.21.15.38:0/963310714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b92543b9d0 0x55b92545be90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6fc9ffb640 0 client.15141 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 >> 172.21.15.38:0/963310714 conn(0x55b925437f60 msgr2=0x55b925444de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 shutdown_connections 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 1 -- 172.21.15.38:0/963310714 wait complete. 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.420+0000 7f6feac1b740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T05:41:00.428 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24496, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 117.518994453, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 89, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 137.377746766 2024-09-19T05:41:00.429 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 msgr2=0x7f6fd00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 0x7f6fd00a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc000c00 tx=0x7f6fcc005b50 comp rx=0 tx=0).stop 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 msgr2=0x7f6fe419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e9f0 tx=0x7f6fdc00edf0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 shutdown_connections 2024-09-19T05:41:00.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6fd00a3350 0x7f6fd00a5810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe410e3c0 0x7f6fe419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe4074f40 0x7f6fe419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 --2- 172.21.15.38:0/1289752665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6fe4074620 0x7f6fe419a6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 >> 172.21.15.38:0/1289752665 conn(0x7f6fe40fd8c0 msgr2=0x7f6fe40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 shutdown_connections 2024-09-19T05:41:00.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:00.421+0000 7f6fe8842640 1 -- 172.21.15.38:0/1289752665 wait complete. 2024-09-19T05:41:01.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:00 smithi038 ceph-mon[31227]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:00 smithi179 ceph-mon[35781]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:01.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:00 smithi047 ceph-mon[35327]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:01.244 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T05:41:01.245 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:41:01.256 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:41:01.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T05:41:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:02.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:02 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:02 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:02 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:02.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- 172.21.15.38:0/120220122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c105010 msgr2=0x7f314c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:02.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 --2- 172.21.15.38:0/120220122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c105010 0x7f314c107400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3134009900 tx=0x7f313402a410 comp rx=0 tx=0).stop 2024-09-19T05:41:02.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- 172.21.15.38:0/120220122 shutdown_connections 2024-09-19T05:41:02.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 --2- 172.21.15.38:0/120220122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c107940 0x7f314c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 --2- 172.21.15.38:0/120220122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c105010 0x7f314c107400 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 --2- 172.21.15.38:0/120220122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c1026e0 0x7f314c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- 172.21.15.38:0/120220122 >> 172.21.15.38:0/120220122 conn(0x7f314c0fc280 msgr2=0x7f314c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- 172.21.15.38:0/120220122 shutdown_connections 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- 172.21.15.38:0/120220122 wait complete. 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 Processor -- start 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.475+0000 7f3151b03640 1 -- start start 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 0x7f314c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c107940 0x7f314c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10d850 con 0x7f314c1026e0 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10d6d0 con 0x7f314c107940 2024-09-19T05:41:02.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3151b03640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10d9d0 con 0x7f314c105010 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53728/0 (socket says 172.21.15.38:53728) 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 -- 172.21.15.38:0/944166294 learned_addr learned my addr 172.21.15.38:0/944166294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3142ffd640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 0x7f314c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314bfff640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c107940 0x7f314c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 -- 172.21.15.38:0/944166294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 msgr2=0x7f314c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 0x7f314c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 -- 172.21.15.38:0/944166294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c107940 msgr2=0x7f314c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:02.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c107940 0x7f314c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 -- 172.21.15.38:0/944166294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c19fb60 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f3142ffd640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 0x7f314c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.476+0000 7f314b7fe640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d950 tx=0x7f313c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.477+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c017030 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.477+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c19fe50 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.477+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f313c004020 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.477+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0047a0 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.477+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1a02b0 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.478+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c0fdd70 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.479+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f313c0041d0 con 0x7f314c1026e0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.480+0000 7f31497fa640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 0x7f31380a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.480+0000 7f3142ffd640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 0x7f31380a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:02.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.480+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f313c0cf940 con 0x7f314c1026e0 2024-09-19T05:41:02.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.480+0000 7f3142ffd640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 0x7f31380a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3134006010 tx=0x7f3134032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:02.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.484+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f313c09c140 con 0x7f314c1026e0 2024-09-19T05:41:02.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.697+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f314c062630 con 0x7f31380a3300 2024-09-19T05:41:02.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.699+0000 7f31497fa640 1 -- 172.21.15.38:0/944166294 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f314c062630 con 0x7f31380a3300 2024-09-19T05:41:02.701 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-09-19T05:41:02.701 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-09-19T05:41:02.701 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 msgr2=0x7f31380a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 0x7f31380a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3134006010 tx=0x7f3134032cf0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 msgr2=0x7f314c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d950 tx=0x7f313c00de10 comp rx=0 tx=0).stop 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 shutdown_connections 2024-09-19T05:41:02.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f31380a3300 0x7f31380a57c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f314c107940 0x7f314c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c105010 0x7f314c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 --2- 172.21.15.38:0/944166294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c1026e0 0x7f314c197f30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 >> 172.21.15.38:0/944166294 conn(0x7f314c0fc280 msgr2=0x7f314c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 shutdown_connections 2024-09-19T05:41:02.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:02.701+0000 7f3151b03640 1 -- 172.21.15.38:0/944166294 wait complete. 2024-09-19T05:41:03.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:03 smithi038 ceph-mon[31227]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:41:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:03 smithi179 ceph-mon[35781]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:41:03.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:03 smithi047 ceph-mon[35327]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:41:03.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T05:41:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.297+0000 7f85ec63b640 1 -- 172.21.15.38:0/1351171190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 msgr2=0x7f85e4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.297+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1351171190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4102000 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009900 tx=0x7f85e002a410 comp rx=0 tx=0).stop 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 -- 172.21.15.38:0/1351171190 shutdown_connections 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1351171190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 0x7f85e410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1351171190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 0x7f85e4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1351171190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4102000 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 -- 172.21.15.38:0/1351171190 >> 172.21.15.38:0/1351171190 conn(0x7f85e40fd390 msgr2=0x7f85e40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 -- 172.21.15.38:0/1351171190 shutdown_connections 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 -- 172.21.15.38:0/1351171190 wait complete. 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 Processor -- start 2024-09-19T05:41:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.298+0000 7f85ec63b640 1 -- start start 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 0x7f85e4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 0x7f85e419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410d860 con 0x7f85e4101c00 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410d6e0 con 0x7f85e4102e00 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410d9e0 con 0x7f85e41037c0 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53738/0 (socket says 172.21.15.38:53738) 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 -- 172.21.15.38:0/1251059221 learned_addr learned my addr 172.21.15.38:0/1251059221 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85e9bd6640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 0x7f85e4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85eabd8640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 0x7f85e419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 -- 172.21.15.38:0/1251059221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 msgr2=0x7f85e419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 0x7f85e419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 -- 172.21.15.38:0/1251059221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 msgr2=0x7f85e4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 0x7f85e4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 -- 172.21.15.38:0/1251059221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e419fb30 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ea3d7640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85e0031610 tx=0x7f85e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e003f030 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e0003ad0 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0035070 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.299+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e419fdc0 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.300+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a01e0 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.301+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e4109000 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.302+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f85e0042030 con 0x7f85e4101c00 2024-09-19T05:41:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.303+0000 7f85db7fe640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 0x7f85d00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.303+0000 7f85e9bd6640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 0x7f85d00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.303+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85e00f0370 con 0x7f85e4101c00 2024-09-19T05:41:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.303+0000 7f85e9bd6640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 0x7f85d00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f85e4199420 tx=0x7f85d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.307+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85e00bc580 con 0x7f85e4101c00 2024-09-19T05:41:04.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:04 smithi179 ceph-mon[35781]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:04.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:04 smithi047 ceph-mon[35327]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:04.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.492+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f85e4062630 con 0x7f85d00a3300 2024-09-19T05:41:04.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:04 smithi038 ceph-mon[31227]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:04.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.494 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.504+0000 7f85db7fe640 1 -- 172.21.15.38:0/1251059221 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f85e4062630 con 0x7f85d00a3300 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.a smithi038 running (2m) 0s ago 2m 17.2M - 19.1.1-361-g2f599a46 d2baa855c067 6cce782df8a5 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.b smithi047 running (2m) 3s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 1451b31d87e9 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.c smithi179 running (2m) 3s ago 2m 21.1M - 19.1.1-361-g2f599a46 d2baa855c067 f9d8e2f9dbed 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.d smithi038 running (2m) 0s ago 2m 19.9M - 19.1.1-361-g2f599a46 d2baa855c067 3241021ccf80 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.e smithi047 running (2m) 3s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 5a2449bb9620 2024-09-19T05:41:04.506 INFO:teuthology.orchestra.run.smithi038.stdout:mds.f smithi179 running (2m) 3s ago 2m 21.3M - 19.1.1-361-g2f599a46 d2baa855c067 a1e8f5fb61cf 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.g smithi038 running (2m) 0s ago 2m 19.9M - 19.1.1-361-g2f599a46 d2baa855c067 6fb71d531d65 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.h smithi047 running (2m) 3s ago 2m 17.8M - 19.1.1-361-g2f599a46 d2baa855c067 3b394193efcc 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.i smithi179 running (2m) 3s ago 2m 27.0M - 19.1.1-361-g2f599a46 d2baa855c067 41d210b9626b 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.j smithi038 running (2m) 0s ago 2m 14.8M - 19.1.1-361-g2f599a46 d2baa855c067 558f3161bf1d 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.k smithi047 running (2m) 3s ago 2m 14.9M - 19.1.1-361-g2f599a46 d2baa855c067 faf15a7c0b19 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mds.l smithi179 running (2m) 3s ago 2m 21.0M - 19.1.1-361-g2f599a46 d2baa855c067 4e7b10391970 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.x smithi038 *:9283,8765 running (9m) 0s ago 9m 511M - 19.1.1-361-g2f599a46 d2baa855c067 f594ec0a8f31 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.y smithi047 *:8443,8765 running (7m) 3s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 e7c91e3aa3f1 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.z smithi179 *:8443,8765 running (7m) 3s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 829bcaf4e836 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mon.a smithi038 running (9m) 0s ago 9m 53.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 edf8305fb6ee 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mon.b smithi047 running (8m) 3s ago 8m 42.3M 2048M 19.1.1-361-g2f599a46 d2baa855c067 caf1f61d5991 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:mon.c smithi179 running (8m) 3s ago 8m 42.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 b347b383754a 2024-09-19T05:41:04.507 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 0s ago 7m 46.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 042cf0242367 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi047 running (7m) 3s ago 7m 47.2M 4273M 19.1.1-361-g2f599a46 d2baa855c067 b379888bff0b 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi179 running (6m) 3s ago 6m 49.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 65bd0334b67a 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi038 running (6m) 0s ago 6m 55.8M 4273M 19.1.1-361-g2f599a46 d2baa855c067 40ba975bb043 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi047 running (6m) 3s ago 5m 43.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 fd63f68774ba 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi179 running (5m) 3s ago 5m 45.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 463879aec1e4 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 smithi038 running (5m) 0s ago 5m 52.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 e58537546f30 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 smithi047 running (4m) 3s ago 4m 47.6M 4273M 19.1.1-361-g2f599a46 d2baa855c067 6d064b23ebbc 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 smithi179 running (4m) 3s ago 4m 52.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 aa43eed2a247 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 smithi038 running (3m) 0s ago 3m 43.1M 4273M 19.1.1-361-g2f599a46 d2baa855c067 2fef1d60679f 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 smithi047 running (3m) 3s ago 3m 44.1M 4273M 19.1.1-361-g2f599a46 d2baa855c067 80e0a31f40c5 2024-09-19T05:41:04.508 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 smithi179 running (3m) 3s ago 3m 41.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 53c1b993e0e6 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 msgr2=0x7f85d00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 0x7f85d00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f85e4199420 tx=0x7f85d4005f90 comp rx=0 tx=0).stop 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 msgr2=0x7f85e4197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85e0031610 tx=0x7f85e0004270 comp rx=0 tx=0).stop 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 shutdown_connections 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3300 0x7f85d00a57c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e41037c0 0x7f85e419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4102e00 0x7f85e4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 --2- 172.21.15.38:0/1251059221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4101c00 0x7f85e4197f00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 >> 172.21.15.38:0/1251059221 conn(0x7f85e40fd390 msgr2=0x7f85e41061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.506+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 shutdown_connections 2024-09-19T05:41:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:04.507+0000 7f85ec63b640 1 -- 172.21.15.38:0/1251059221 wait complete. 2024-09-19T05:41:05.228 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T05:41:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:05 smithi179 ceph-mon[35781]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:41:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:41:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:05.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:05 smithi179 ceph-mon[35781]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:05 smithi047 ceph-mon[35327]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:41:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:41:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:05 smithi047 ceph-mon[35327]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:05.488 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:05.609 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:05 smithi038 ceph-mon[31227]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:05.609 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:41:05.609 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:41:05.609 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:41:05.609 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:05 smithi038 ceph-mon[31227]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 -- 172.21.15.38:0/4235835519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 msgr2=0x7f9058074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 --2- 172.21.15.38:0/4235835519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f9058074600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9048009900 tx=0x7f904802a410 comp rx=0 tx=0).stop 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 -- 172.21.15.38:0/4235835519 shutdown_connections 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 --2- 172.21.15.38:0/4235835519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058109a70 0x7f905810bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 --2- 172.21.15.38:0/4235835519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058074b40 0x7f9058074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 --2- 172.21.15.38:0/4235835519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f9058074600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 -- 172.21.15.38:0/4235835519 >> 172.21.15.38:0/4235835519 conn(0x7f90580fd390 msgr2=0x7f90580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 -- 172.21.15.38:0/4235835519 shutdown_connections 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.997+0000 7f905da8b640 1 -- 172.21.15.38:0/4235835519 wait complete. 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 Processor -- start 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 -- start start 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058074b40 0x7f905819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 0x7f90581a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f905810fad0 con 0x7f90580761a0 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f905810f950 con 0x7f9058074b40 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f905da8b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f905810fc50 con 0x7f9058109a70 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f9056ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f9056ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53756/0 (socket says 172.21.15.38:53756) 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f9056ffd640 1 -- 172.21.15.38:0/3886781206 learned_addr learned my addr 172.21.15.38:0/3886781206 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:06.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.998+0000 7f90577fe640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058074b40 0x7f905819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9057fff640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 0x7f90581a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 -- 172.21.15.38:0/3886781206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 msgr2=0x7f90581a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 0x7f90581a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 -- 172.21.15.38:0/3886781206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058074b40 msgr2=0x7f905819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058074b40 0x7f905819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 -- 172.21.15.38:0/3886781206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581a1cf0 con 0x7f90580761a0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9057fff640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 0x7f90581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9056ffd640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b4a0 tx=0x7f904c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c014070 con 0x7f90580761a0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581a1fe0 con 0x7f90580761a0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904c010af0 con 0x7f90580761a0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c017d10 con 0x7f90580761a0 2024-09-19T05:41:06.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:05.999+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581a2510 con 0x7f90580761a0 2024-09-19T05:41:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.000+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9058108880 con 0x7f90580761a0 2024-09-19T05:41:06.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.001+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f904c017070 con 0x7f90580761a0 2024-09-19T05:41:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.002+0000 7f9054ff9640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 0x7f90400a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.003+0000 7f90577fe640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 0x7f90400a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.003+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f904c0cf3a0 con 0x7f90580761a0 2024-09-19T05:41:06.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.003+0000 7f90577fe640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 0x7f90400a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90480314e0 tx=0x7f9048000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:06.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.006+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f904c09bd50 con 0x7f90580761a0 2024-09-19T05:41:06.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.189+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9058062630 con 0x7f90400a3300 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.195+0000 7f9054ff9640 1 -- 172.21.15.38:0/3886781206 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9058062630 con 0x7f90400a3300 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stdout:mds.all 12/12 5s ago 2m smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 3/3 5s ago 8m smithi038=x;smithi047=y;smithi179=z;count:3 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stdout:mon 3/3 5s ago 8m smithi038:172.21.15.38=a;smithi047:172.21.15.47=b;smithi179:172.21.15.179=c;count:3 2024-09-19T05:41:06.197 INFO:teuthology.orchestra.run.smithi038.stdout:osd 12 5s ago - 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 msgr2=0x7f90400a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 0x7f90400a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90480314e0 tx=0x7f9048000bc0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 msgr2=0x7f905819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b4a0 tx=0x7f904c00b960 comp rx=0 tx=0).stop 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 shutdown_connections 2024-09-19T05:41:06.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f90400a3300 0x7f90400a57c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9058109a70 0x7f90581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580761a0 0x7f905819a600 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 --2- 172.21.15.38:0/3886781206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9058074b40 0x7f905819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 >> 172.21.15.38:0/3886781206 conn(0x7f90580fd390 msgr2=0x7f9058105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 shutdown_connections 2024-09-19T05:41:06.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:06.197+0000 7f905da8b640 1 -- 172.21.15.38:0/3886781206 wait complete. 2024-09-19T05:41:06.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T05:41:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:07.447 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:07 smithi038 ceph-mon[31227]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:41:07.447 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:07 smithi038 ceph-mon[31227]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:07 smithi179 ceph-mon[35781]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:41:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:07 smithi179 ceph-mon[35781]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:07.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:07 smithi047 ceph-mon[35327]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:41:07.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:07 smithi047 ceph-mon[35327]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2619761069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 msgr2=0x7fa854102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2619761069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854102150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa848009900 tx=0x7fa84802a410 comp rx=0 tx=0).stop 2024-09-19T05:41:07.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2619761069 shutdown_connections 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2619761069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 0x7fa854108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2619761069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854102150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2619761069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 0x7fa854100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2619761069 >> 172.21.15.38:0/2619761069 conn(0x7fa8540fc280 msgr2=0x7fa8540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2619761069 shutdown_connections 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.668+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2619761069 wait complete. 2024-09-19T05:41:07.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 Processor -- start 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 -- start start 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 0x7fa854197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 0x7fa85419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa85410c750 con 0x7fa854101cd0 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa85410c5d0 con 0x7fa854102690 2024-09-19T05:41:07.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa85a0ce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa85410c8d0 con 0x7fa854100ad0 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa852ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa852ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59860/0 (socket says 172.21.15.38:59860) 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa852ffd640 1 -- 172.21.15.38:0/2896813092 learned_addr learned my addr 172.21.15.38:0/2896813092 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa853fff640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 0x7fa85419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.669+0000 7fa8537fe640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 0x7fa854197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 -- 172.21.15.38:0/2896813092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 msgr2=0x7fa854197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:07.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 0x7fa854197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 -- 172.21.15.38:0/2896813092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 msgr2=0x7fa85419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:07.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 0x7fa85419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 -- 172.21.15.38:0/2896813092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85419fc00 con 0x7fa854101cd0 2024-09-19T05:41:07.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa852ffd640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa848006010 tx=0x7fa8480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:07.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84803f030 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa848003ce0 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85419fe90 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848035070 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.670+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541a02f0 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.671+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8541a0520 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.672+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa848042030 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.673+0000 7fa850ff9640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 0x7fa83c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.673+0000 7fa8537fe640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 0x7fa83c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.673+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa8480f0170 con 0x7fa854101cd0 2024-09-19T05:41:07.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.674+0000 7fa8537fe640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 0x7fa83c0a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa854101b30 tx=0x7fa844009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:07.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.677+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8480bc320 con 0x7fa854101cd0 2024-09-19T05:41:07.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.858+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa85406a710 con 0x7fa83c0a3380 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.859+0000 7fa850ff9640 1 -- 172.21.15.38:0/2896813092 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa85406a710 con 0x7fa83c0a3380 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stdout:smithi047 172.21.15.47 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stdout:smithi179 172.21.15.179 2024-09-19T05:41:07.861 INFO:teuthology.orchestra.run.smithi038.stdout:3 hosts in cluster 2024-09-19T05:41:07.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 msgr2=0x7fa83c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 0x7fa83c0a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa854101b30 tx=0x7fa844009500 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 msgr2=0x7fa854198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa848006010 tx=0x7fa8480042f0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 shutdown_connections 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa83c0a3380 0x7fa83c0a5840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa854102690 0x7fa85419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa854101cd0 0x7fa854198510 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 --2- 172.21.15.38:0/2896813092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa854100ad0 0x7fa854197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 >> 172.21.15.38:0/2896813092 conn(0x7fa8540fc280 msgr2=0x7fa85406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 shutdown_connections 2024-09-19T05:41:07.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:07.862+0000 7fa85a0ce640 1 -- 172.21.15.38:0/2896813092 wait complete. 2024-09-19T05:41:08.605 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T05:41:08.865 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:09.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:09 smithi038 ceph-mon[31227]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:09.279 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:09 smithi038 ceph-mon[31227]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 -- 172.21.15.38:0/3405742107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb40ffed0 msgr2=0x7f0fb41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 --2- 172.21.15.38:0/3405742107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb40ffed0 0x7f0fb41002d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009930 tx=0x7f0fa402a420 comp rx=0 tx=0).stop 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 -- 172.21.15.38:0/3405742107 shutdown_connections 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 --2- 172.21.15.38:0/3405742107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb4103870 0x7f0fb410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 --2- 172.21.15.38:0/3405742107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 0x7f0fb4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 --2- 172.21.15.38:0/3405742107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb40ffed0 0x7f0fb41002d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 -- 172.21.15.38:0/3405742107 >> 172.21.15.38:0/3405742107 conn(0x7f0fb40fba40 msgr2=0x7f0fb40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 -- 172.21.15.38:0/3405742107 shutdown_connections 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.362+0000 7f0fb9243640 1 -- 172.21.15.38:0/3405742107 wait complete. 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 Processor -- start 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 -- start start 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb40ffed0 0x7f0fb4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 0x7f0fb4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410d890 con 0x7f0fb4103870 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410d710 con 0x7f0fb4102eb0 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb9243640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410da10 con 0x7f0fb40ffed0 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59880/0 (socket says 172.21.15.38:59880) 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb359e640 1 -- 172.21.15.38:0/1682959469 learned_addr learned my addr 172.21.15.38:0/1682959469 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb2d9d640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb40ffed0 0x7f0fb4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.363+0000 7f0fb259c640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 0x7f0fb4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 -- 172.21.15.38:0/1682959469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb40ffed0 msgr2=0x7f0fb4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb40ffed0 0x7f0fb4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 -- 172.21.15.38:0/1682959469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 msgr2=0x7f0fb4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 0x7f0fb4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 -- 172.21.15.38:0/1682959469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb419fc00 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb359e640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00cad0 tx=0x7f0fac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00ae20 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fac005020 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb419fef0 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00f6f0 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.364+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41a0350 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.365+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb41091f0 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.366+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0fac0051d0 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.367+0000 7f0fabfff640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 0x7f0fa00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.368+0000 7f0fb2d9d640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 0x7f0fa00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.368+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0fac0cfd20 con 0x7f0fb4103870 2024-09-19T05:41:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.368+0000 7f0fb2d9d640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 0x7f0fa00a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009900 tx=0x7f0fa40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.372+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0fac014030 con 0x7f0fb4103870 2024-09-19T05:41:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:09 smithi179 ceph-mon[35781]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:09 smithi179 ceph-mon[35781]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:09.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:09 smithi047 ceph-mon[35327]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:09 smithi047 ceph-mon[35327]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.554+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0fb4062630 con 0x7f0fa00a3300 2024-09-19T05:41:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.556+0000 7f0fabfff640 1 -- 172.21.15.38:0/1682959469 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0fb4062630 con 0x7f0fa00a3300 2024-09-19T05:41:09.558 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T05:41:09.558 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:41:09.558 INFO:teuthology.orchestra.run.smithi038.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:41:09.558 INFO:teuthology.orchestra.run.smithi038.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:41:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.558+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 msgr2=0x7f0fa00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.558+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 0x7f0fa00a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009900 tx=0x7f0fa40315b0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.558+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 msgr2=0x7f0fb419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.558+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00cad0 tx=0x7f0fac00cf90 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 shutdown_connections 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0fa00a3300 0x7f0fa00a57c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fb4103870 0x7f0fb419f590 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb4102eb0 0x7f0fb4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 --2- 172.21.15.38:0/1682959469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fb40ffed0 0x7f0fb4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 >> 172.21.15.38:0/1682959469 conn(0x7f0fb40fba40 msgr2=0x7f0fb41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 shutdown_connections 2024-09-19T05:41:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:09.559+0000 7f0fb9243640 1 -- 172.21.15.38:0/1682959469 wait complete. 2024-09-19T05:41:10.310 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:41:10.319 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:41:10.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T05:41:10.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:10 smithi179 ceph-mon[35781]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:10.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T05:41:10.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T05:41:10.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T05:41:10.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:10 smithi047 ceph-mon[35327]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:10.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T05:41:10.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T05:41:10.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T05:41:10.582 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:10.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:10 smithi038 ceph-mon[31227]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:41:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T05:41:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T05:41:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-09-19T05:41:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 -- 172.21.15.38:0/2636854474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8104f90 msgr2=0x7f8dc8107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/2636854474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8104f90 0x7f8dc8107380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009900 tx=0x7f8db002a410 comp rx=0 tx=0).stop 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 -- 172.21.15.38:0/2636854474 shutdown_connections 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/2636854474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc81078c0 0x7f8dc8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/2636854474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8104f90 0x7f8dc8107380 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/2636854474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 -- 172.21.15.38:0/2636854474 >> 172.21.15.38:0/2636854474 conn(0x7f8dc80fc280 msgr2=0x7f8dc80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 -- 172.21.15.38:0/2636854474 shutdown_connections 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.106+0000 7f8dcdc47640 1 -- 172.21.15.38:0/2636854474 wait complete. 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 Processor -- start 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 -- start start 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc8104f90 0x7f8dc8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810d860 con 0x7f8dc81078c0 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810d6e0 con 0x7f8dc8102660 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dcdc47640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810d9e0 con 0x7f8dc8104f90 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59906/0 (socket says 172.21.15.38:59906) 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 -- 172.21.15.38:0/3929273069 learned_addr learned my addr 172.21.15.38:0/3929273069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dbeffd640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc8104f90 0x7f8dc8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc77fe640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 -- 172.21.15.38:0/3929273069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc8104f90 msgr2=0x7f8dc8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc8104f90 0x7f8dc8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 -- 172.21.15.38:0/3929273069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 msgr2=0x7f8dc8197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc7fff640 1 -- 172.21.15.38:0/3929273069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc819fb40 con 0x7f8dc81078c0 2024-09-19T05:41:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.107+0000 7f8dc77fe640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dc7fff640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000eb10 tx=0x7f8dc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0004b20 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc0004cc0 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc819fe30 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0010760 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.108+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81a02e0 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.109+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc80fdd70 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.110+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8dc0005020 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.111+0000 7f8dc57fa640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 0x7f8db40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.111+0000 7f8dc77fe640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 0x7f8db40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.111+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9187+0+0 (secure 0 0 0) 0x7f8dc00d0200 con 0x7f8dc81078c0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.111+0000 7f8dc77fe640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 0x7f8db40a5730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8db8005d00 tx=0x7f8db8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.115+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8dc009cb60 con 0x7f8dc81078c0 2024-09-19T05:41:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.335+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8dc802d7c0 con 0x7f8dc81078c0 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:41:11.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:11 smithi038 ceph-mon[31227]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.335+0000 7f8dc57fa640 1 -- 172.21.15.38:0/3929273069 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3128 (secure 0 0 0) 0x7f8dc009c460 con 0x7f8dc81078c0 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:e27 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:btime 2024-09-19T05:39:48:102916+0000 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:11.339 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 27 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-09-19T05:39:02.544314+0000 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-09-19T05:39:48.102894+0000 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 300 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-09-19T05:41:11.340 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 5 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1,2,3,4 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24496,1=24485,2=24481,3=24475,4=24455} 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3,4] 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-09-19T05:41:11.341 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 24496 members: 24496,24485,24475,24481,24455 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.i{0:24496} state up:active seq 6 addr [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.l{0:24436} state up:standby-replay seq 1 addr [v2:172.21.15.179:6832/2514682242,v1:172.21.15.179:6833/2514682242] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b{1:24485} state up:active seq 15 addr [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.e{1:24434} state up:standby-replay seq 1 addr [v2:172.21.15.47:6834/3758113926,v1:172.21.15.47:6835/3758113926] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c{2:24481} state up:active seq 18 addr [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a{2:14610} state up:standby-replay seq 1 addr [v2:172.21.15.38:6840/3698066275,v1:172.21.15.38:6841/3698066275] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.f{3:24475} state up:active seq 19 addr [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.g{3:14598} state up:standby-replay seq 1 addr [v2:172.21.15.38:6838/2963619728,v1:172.21.15.38:6839/2963619728] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.h{4:24455} state up:active seq 18 addr [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d{4:14547} state up:standby-replay seq 1 addr [v2:172.21.15.38:6836/4064878755,v1:172.21.15.38:6837/4064878755] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.k{-1:14529} state up:standby seq 1 addr [v2:172.21.15.47:6832/418551929,v1:172.21.15.47:6833/418551929] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.38:6834/1823768256,v1:172.21.15.38:6835/1823768256] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 msgr2=0x7f8db40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 0x7f8db40a5730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8db8005d00 tx=0x7f8db8005c70 comp rx=0 tx=0).stop 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 msgr2=0x7f8dc819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000eb10 tx=0x7f8dc000c2d0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 shutdown_connections 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8db40a3270 0x7f8db40a5730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc81078c0 0x7f8dc819f4d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc8104f90 0x7f8dc8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 --2- 172.21.15.38:0/3929273069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8dc8102660 0x7f8dc8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 >> 172.21.15.38:0/3929273069 conn(0x7f8dc80fc280 msgr2=0x7f8dc8103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 shutdown_connections 2024-09-19T05:41:11.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:11.339+0000 7f8dcdc47640 1 -- 172.21.15.38:0/3929273069 wait complete. 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:41:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:11 smithi047 ceph-mon[35327]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:41:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T05:41:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T05:41:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-09-19T05:41:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:41:11.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:11 smithi179 ceph-mon[35781]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:41:12.059 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T05:41:12.320 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:12.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:12 smithi038 ceph-mon[31227]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:41:12.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:12 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3929273069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:41:12.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:12 smithi047 ceph-mon[35327]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:41:12.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:12 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3929273069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:41:12.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:12 smithi179 ceph-mon[35781]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:41:12.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:12 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3929273069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:41:12.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.847+0000 7f9250d6a640 1 -- 172.21.15.38:0/2854821712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c102690 msgr2=0x7f924c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:12.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.847+0000 7f9250d6a640 1 --2- 172.21.15.38:0/2854821712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c102690 0x7f924c108f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9244009ca0 tx=0x7f924402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 -- 172.21.15.38:0/2854821712 shutdown_connections 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 --2- 172.21.15.38:0/2854821712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c102690 0x7f924c108f20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 --2- 172.21.15.38:0/2854821712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c101cd0 0x7f924c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 --2- 172.21.15.38:0/2854821712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 0x7f924c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 -- 172.21.15.38:0/2854821712 >> 172.21.15.38:0/2854821712 conn(0x7f924c0fc280 msgr2=0x7f924c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 -- 172.21.15.38:0/2854821712 shutdown_connections 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 -- 172.21.15.38:0/2854821712 wait complete. 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 Processor -- start 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.848+0000 7f9250d6a640 1 -- start start 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 0x7f924c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:12.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c102690 0x7f924c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f924c10c730 con 0x7f924c101cd0 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f924c10c5b0 con 0x7f924c102690 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f924c10c8b0 con 0x7f924c100ad0 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59928/0 (socket says 172.21.15.38:59928) 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 -- 172.21.15.38:0/3457268198 learned_addr learned my addr 172.21.15.38:0/3457268198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f924ad9d640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c102690 0x7f924c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f924a59c640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 0x7f924c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 -- 172.21.15.38:0/3457268198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 msgr2=0x7f924c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 0x7f924c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 -- 172.21.15.38:0/3457268198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c102690 msgr2=0x7f924c1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c102690 0x7f924c1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 -- 172.21.15.38:0/3457268198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1a1cc0 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9249d9b640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f924000e9f0 tx=0x7f924000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924000cd80 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9240010910 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.849+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c1a1fb0 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.850+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240018990 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.850+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1a24b0 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.851+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c1a26e0 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.852+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9240010ac0 con 0x7f924c101cd0 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.853+0000 7f923b7fe640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 0x7f92340a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:12.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.853+0000 7f924a59c640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 0x7f92340a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:12.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.853+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f92400cfea0 con 0x7f924c101cd0 2024-09-19T05:41:12.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.853+0000 7f924a59c640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 0x7f92340a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f923c005fd0 tx=0x7f923c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:12.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:12.857+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f924009c1b0 con 0x7f924c101cd0 2024-09-19T05:41:13.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.033+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f924c1a2a70 con 0x7f924c101cd0 2024-09-19T05:41:13.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.034+0000 7f923b7fe640 1 -- 172.21.15.38:0/3457268198 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5673 (secure 0 0 0) 0x7f92400a04b0 con 0x7f924c101cd0 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 91 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:fsid 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-09-19T05:31:23.688213+0000 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-09-19T05:41:11.190669+0000 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:crush_version 27 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:full_ratio 0.95 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:backfillfull_ratio 0.9 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:nearfull_ratio 0.85 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:require_min_compat_client luminous 2024-09-19T05:41:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:min_compat_client luminous 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:require_osd_release squid 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:stretch_mode_enabled false 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:max_osd 12 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6802/3037922100,v1:172.21.15.38:6803/3037922100] [v2:172.21.15.38:6804/3037922100,v1:172.21.15.38:6805/3037922100] exists,up 6d08454b-46b5-4ad6-8e7f-3c79c155e73d 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6800/716320607,v1:172.21.15.47:6801/716320607] [v2:172.21.15.47:6802/716320607,v1:172.21.15.47:6803/716320607] exists,up 7156f9e6-19c1-4422-89f5-97209c281838 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/4020882332,v1:172.21.15.179:6801/4020882332] [v2:172.21.15.179:6802/4020882332,v1:172.21.15.179:6803/4020882332] exists,up 94fbc241-9df4-4189-9369-72b359a64f08 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6810/1962488545,v1:172.21.15.38:6811/1962488545] [v2:172.21.15.38:6812/1962488545,v1:172.21.15.38:6813/1962488545] exists,up 8c145d0a-468b-40cd-bc68-2214559e2fdf 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 up in weight 1 up_from 32 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6808/3118152095,v1:172.21.15.47:6809/3118152095] [v2:172.21.15.47:6810/3118152095,v1:172.21.15.47:6811/3118152095] exists,up 2ea25802-a31e-4e01-bccf-183ee703afe8 2024-09-19T05:41:13.038 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 up in weight 1 up_from 37 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/322546150,v1:172.21.15.179:6809/322546150] [v2:172.21.15.179:6810/322546150,v1:172.21.15.179:6811/322546150] exists,up 03dfe06a-5da0-4925-a09a-9fea434b0633 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 up in weight 1 up_from 44 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6818/2208960184,v1:172.21.15.38:6819/2208960184] [v2:172.21.15.38:6820/2208960184,v1:172.21.15.38:6821/2208960184] exists,up e40f682d-ec98-470b-b2e2-19946849a6f0 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 up in weight 1 up_from 49 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6816/2705740748,v1:172.21.15.47:6817/2705740748] [v2:172.21.15.47:6818/2705740748,v1:172.21.15.47:6819/2705740748] exists,up a99cd83f-58cb-47b9-bcea-35e82ceacbc8 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 up in weight 1 up_from 56 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/2844755056,v1:172.21.15.179:6817/2844755056] [v2:172.21.15.179:6818/2844755056,v1:172.21.15.179:6819/2844755056] exists,up 1306f615-40c3-4be2-b882-b665b1b01588 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 up in weight 1 up_from 63 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6826/3187320508,v1:172.21.15.38:6827/3187320508] [v2:172.21.15.38:6828/3187320508,v1:172.21.15.38:6829/3187320508] exists,up f3874c68-e48a-452f-a6f4-c349235e29da 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 up in weight 1 up_from 68 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6824/1633858254,v1:172.21.15.47:6825/1633858254] [v2:172.21.15.47:6826/1633858254,v1:172.21.15.47:6827/1633858254] exists,up 1a1ddce0-f377-454c-a5f2-b8ecad532909 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 up in weight 1 up_from 75 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/1092638646,v1:172.21.15.179:6825/1092638646] [v2:172.21.15.179:6826/1092638646,v1:172.21.15.179:6827/1092638646] exists,up 7cda5e25-4843-426b-a917-89f6546f30a8 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.5 [8,11] 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.c [2,4] 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.1d [8,11,2,1] 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.0 [4,8] 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.c [3,9] 2024-09-19T05:41:13.039 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.11 [4,6] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.1a [5,7] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.22 [3,9] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.34 [4,9] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.a [5,8] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.10 [9,10] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.1b [9,8] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.1c [9,8] 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/3555821050 expires 2024-09-20T05:32:09.496697+0000 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2329208823 expires 2024-09-20T05:32:09.496697+0000 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1466442042 expires 2024-09-20T05:32:09.496697+0000 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/3555821050 expires 2024-09-20T05:32:09.496697+0000 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1122120616 expires 2024-09-20T05:31:45.752428+0000 2024-09-19T05:41:13.040 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2032678289 expires 2024-09-20T05:31:45.752428+0000 2024-09-19T05:41:13.041 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2581276558 expires 2024-09-20T05:31:45.752428+0000 2024-09-19T05:41:13.041 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/4202404879 expires 2024-09-20T05:31:45.752428+0000 2024-09-19T05:41:13.041 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2319061439 expires 2024-09-20T05:32:09.496697+0000 2024-09-19T05:41:13.041 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/4202404879 expires 2024-09-20T05:31:45.752428+0000 2024-09-19T05:41:13.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 msgr2=0x7f92340a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 0x7f92340a57b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f923c005fd0 tx=0x7f923c00b410 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 msgr2=0x7f924c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f924000e9f0 tx=0x7f924000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 shutdown_connections 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f92340a32f0 0x7f92340a57b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f924c102690 0x7f924c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f924c101cd0 0x7f924c19a5d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 --2- 172.21.15.38:0/3457268198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924c100ad0 0x7f924c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 >> 172.21.15.38:0/3457268198 conn(0x7f924c0fc280 msgr2=0x7f924c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 shutdown_connections 2024-09-19T05:41:13.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:13.036+0000 7f9250d6a640 1 -- 172.21.15.38:0/3457268198 wait complete. 2024-09-19T05:41:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:13 smithi038 ceph-mon[31227]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T05:41:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:13 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3457268198' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:41:13.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:13 smithi047 ceph-mon[35327]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T05:41:13.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:13 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3457268198' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:41:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:13 smithi179 ceph-mon[35781]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-19T05:41:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:13 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3457268198' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:41:13.716 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T05:41:13.728 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:41:13.986 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:14.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 -- 172.21.15.38:0/2464651102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 msgr2=0x7f85e410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:14.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 --2- 172.21.15.38:0/2464651102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e410ea90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009ca0 tx=0x7f85dc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 -- 172.21.15.38:0/2464651102 shutdown_connections 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 --2- 172.21.15.38:0/2464651102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e410ea90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 --2- 172.21.15.38:0/2464651102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 0x7f85e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 --2- 172.21.15.38:0/2464651102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 -- 172.21.15.38:0/2464651102 >> 172.21.15.38:0/2464651102 conn(0x7f85e40fd8c0 msgr2=0x7f85e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 -- 172.21.15.38:0/2464651102 shutdown_connections 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.478+0000 7f85e9055640 1 -- 172.21.15.38:0/2464651102 wait complete. 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 Processor -- start 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 -- start start 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 0x7f85e4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:14.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4111f70 con 0x7f85e4106f10 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4111df0 con 0x7f85e4101de0 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.479+0000 7f85e9055640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41120f0 con 0x7f85e4102790 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59942/0 (socket says 172.21.15.38:59942) 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 -- 172.21.15.38:0/1532004442 learned_addr learned my addr 172.21.15.38:0/1532004442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85da59c640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 0x7f85e4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e2d9d640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 -- 172.21.15.38:0/1532004442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 msgr2=0x7f85e4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 0x7f85e4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 -- 172.21.15.38:0/1532004442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 msgr2=0x7f85e41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 -- 172.21.15.38:0/1532004442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e419d3a0 con 0x7f85e4106f10 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e2d9d640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e359e640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85dc031a20 tx=0x7f85dc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc003c30 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e419d630 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85dc003dd0 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc02fcf0 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.480+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a4f70 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.481+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e41021c0 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.483+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f85dc004b40 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.483+0000 7f85e0d99640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.484+0000 7f85e2d9d640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.484+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f85dc0f1190 con 0x7f85e4106f10 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.484+0000 7f85e2d9d640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 0x7f85d00a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005fd0 tx=0x7f85d4005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.487+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85dc0bdb80 con 0x7f85e4106f10 2024-09-19T05:41:14.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.701+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85e410a150 con 0x7f85e4106f10 2024-09-19T05:41:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.702+0000 7f85e0d99640 1 -- 172.21.15.38:0/1532004442 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7f85dc02fe90 con 0x7f85e4106f10 2024-09-19T05:41:14.704 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:14.705 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:41:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 msgr2=0x7f85d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 0x7f85d00a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005fd0 tx=0x7f85d4005ea0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 msgr2=0x7f85e419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f85dc031a20 tx=0x7f85dc031a50 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 shutdown_connections 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f85d00a3240 0x7f85d00a5700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85e4106f10 0x7f85e419ccc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85e4102790 0x7f85e4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 --2- 172.21.15.38:0/1532004442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85e4101de0 0x7f85e41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 >> 172.21.15.38:0/1532004442 conn(0x7f85e40fd8c0 msgr2=0x7f85e40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 shutdown_connections 2024-09-19T05:41:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:14.704+0000 7f85e9055640 1 -- 172.21.15.38:0/1532004442 wait complete. 2024-09-19T05:41:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:15 smithi038 ceph-mon[31227]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:15 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1532004442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:15.430 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:41:15.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:15 smithi047 ceph-mon[35327]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:15 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1532004442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:15.690 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:15 smithi179 ceph-mon[35781]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:15 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1532004442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 -- 172.21.15.38:0/1249709270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374106a80 msgr2=0x7f6374106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 --2- 172.21.15.38:0/1249709270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374106a80 0x7f6374106e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6364009930 tx=0x7f636402a420 comp rx=0 tx=0).stop 2024-09-19T05:41:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 -- 172.21.15.38:0/1249709270 shutdown_connections 2024-09-19T05:41:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 --2- 172.21.15.38:0/1249709270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374101420 0x7f637410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 --2- 172.21.15.38:0/1249709270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 0x7f6374100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 --2- 172.21.15.38:0/1249709270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374106a80 0x7f6374106e60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 -- 172.21.15.38:0/1249709270 >> 172.21.15.38:0/1249709270 conn(0x7f63740fc7a0 msgr2=0x7f63740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 -- 172.21.15.38:0/1249709270 shutdown_connections 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.184+0000 7f6378cb3640 1 -- 172.21.15.38:0/1249709270 wait complete. 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 Processor -- start 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 -- start start 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 0x7f6374198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374106a80 0x7f637419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6374110e80 con 0x7f6374101420 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6374110d00 con 0x7f6374100a80 2024-09-19T05:41:16.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6378cb3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6374111000 con 0x7f6374106a80 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59962/0 (socket says 172.21.15.38:59962) 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 -- 172.21.15.38:0/3753751981 learned_addr learned my addr 172.21.15.38:0/3753751981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f6372d9d640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374106a80 0x7f637419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f637259c640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 0x7f6374198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 -- 172.21.15.38:0/3753751981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374106a80 msgr2=0x7f637419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374106a80 0x7f637419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 -- 172.21.15.38:0/3753751981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 msgr2=0x7f6374198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 0x7f6374198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.185+0000 7f636bfff640 1 -- 172.21.15.38:0/3753751981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419d3f0 con 0x7f6374101420 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f636bfff640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f635c00d950 tx=0x7f635c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c017070 con 0x7f6374101420 2024-09-19T05:41:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419d6e0 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c009c30 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c00f6b0 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.186+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a4fc0 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.187+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6374100ee0 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.188+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f635c0109d0 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.189+0000 7f636b7fe640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 0x7f63600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.189+0000 7f637259c640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 0x7f63600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.189+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f635c0cf920 con 0x7f6374101420 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.189+0000 7f637259c640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 0x7f63600a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.193+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f635c009de0 con 0x7f6374101420 2024-09-19T05:41:16.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.410+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f637402d7c0 con 0x7f6374101420 2024-09-19T05:41:16.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.411+0000 7f636b7fe640 1 -- 172.21.15.38:0/3753751981 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11971 (secure 0 0 0) 0x7f635c09c280 con 0x7f6374101420 2024-09-19T05:41:16.414 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:16.414 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"btime":"2024-09-19T05:39:48:102916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:39:48.102894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24485,24475,24481,24455]},"id":1}]} 2024-09-19T05:41:16.415 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-09-19T05:41:16.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 msgr2=0x7f63600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:16.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 0x7f63600a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364005a20 comp rx=0 tx=0).stop 2024-09-19T05:41:16.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 msgr2=0x7f6374198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:16.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f635c00d950 tx=0x7f635c00de10 comp rx=0 tx=0).stop 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 shutdown_connections 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f63600a3240 0x7f63600a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374106a80 0x7f637419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6374101420 0x7f6374198980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 --2- 172.21.15.38:0/3753751981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6374100a80 0x7f6374198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.413+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 >> 172.21.15.38:0/3753751981 conn(0x7f63740fc7a0 msgr2=0x7f637406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.414+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 shutdown_connections 2024-09-19T05:41:16.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:16.414+0000 7f6378cb3640 1 -- 172.21.15.38:0/3753751981 wait complete. 2024-09-19T05:41:17.092 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:41:17.355 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:17.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:17 smithi047 ceph-mon[35327]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:17 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3753751981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:17.476 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:17 smithi038 ceph-mon[31227]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:17.476 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3753751981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:17 smithi179 ceph-mon[35781]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:17 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3753751981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:17.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 -- 172.21.15.38:0/1318899447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 msgr2=0x7f004c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:17.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 --2- 172.21.15.38:0/1318899447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c10d950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0044009cd0 tx=0x7f004402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 -- 172.21.15.38:0/1318899447 shutdown_connections 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 --2- 172.21.15.38:0/1318899447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c10d950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 --2- 172.21.15.38:0/1318899447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 --2- 172.21.15.38:0/1318899447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 0x7f004c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.850+0000 7f0050d5a640 1 -- 172.21.15.38:0/1318899447 >> 172.21.15.38:0/1318899447 conn(0x7f004c0fc7a0 msgr2=0x7f004c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.851+0000 7f0050d5a640 1 -- 172.21.15.38:0/1318899447 shutdown_connections 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.851+0000 7f0050d5a640 1 -- 172.21.15.38:0/1318899447 wait complete. 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.851+0000 7f0050d5a640 1 Processor -- start 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.851+0000 7f0050d5a640 1 -- start start 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 0x7f004c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f004c110e80 con 0x7f004c101420 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f004c110d00 con 0x7f004c106a80 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f004c111000 con 0x7f004c100a80 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41144/0 (socket says 172.21.15.38:41144) 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 -- 172.21.15.38:0/2227440756 learned_addr learned my addr 172.21.15.38:0/2227440756 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f004a59c640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f004ad9d640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 0x7f004c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 -- 172.21.15.38:0/2227440756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 msgr2=0x7f004c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 -- 172.21.15.38:0/2227440756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 msgr2=0x7f004c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:17.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 0x7f004c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 -- 172.21.15.38:0/2227440756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c19d3d0 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f004a59c640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0043fff640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f003400d770 tx=0x7f003400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034017070 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034005500 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c19d6c0 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.852+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034004e00 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.853+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c1a4fa0 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.854+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c100ee0 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.855+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0034004fa0 con 0x7f004c101420 2024-09-19T05:41:17.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.855+0000 7f00437fe640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 0x7f00380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.856+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f00340cf800 con 0x7f004c101420 2024-09-19T05:41:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.856+0000 7f004a59c640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 0x7f00380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.856+0000 7f004a59c640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 0x7f00380a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f003c005e10 tx=0x7f003c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:17.860+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f003409c160 con 0x7f004c101420 2024-09-19T05:41:18.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.043+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f004c10a020 con 0x7f004c101420 2024-09-19T05:41:18.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.045+0000 7f00437fe640 1 -- 172.21.15.38:0/2227440756 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24875 (secure 0 0 0) 0x7f003409ba60 con 0x7f004c101420 2024-09-19T05:41:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:18.049 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:41:11.190669+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:41:18.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 msgr2=0x7f00380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:18.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 0x7f00380a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f003c005e10 tx=0x7f003c005da0 comp rx=0 tx=0).stop 2024-09-19T05:41:18.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 msgr2=0x7f004c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f003400d770 tx=0x7f003400dc30 comp rx=0 tx=0).stop 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 shutdown_connections 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f00380a3240 0x7f00380a5700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c106a80 0x7f004c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f004c101420 0x7f004c198960 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:18.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 --2- 172.21.15.38:0/2227440756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f004c100a80 0x7f004c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 >> 172.21.15.38:0/2227440756 conn(0x7f004c0fc7a0 msgr2=0x7f004c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 shutdown_connections 2024-09-19T05:41:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:18.047+0000 7f0050d5a640 1 -- 172.21.15.38:0/2227440756 wait complete. 2024-09-19T05:41:18.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:18 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2227440756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:18.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:18 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2227440756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:18.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:18 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2227440756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:18.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:39:02.544314+0000', 'modified': '2024-09-19T05:39:48.102894+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24496, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14547': {'gid': 14547, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/4064878755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 4064878755}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 4064878755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/2963619728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 2963619728}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 2963619728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6841/3698066275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 3698066275}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 3698066275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/3758113926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3758113926}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3758113926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/2514682242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2514682242}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 2514682242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.47:6837/2962930600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2962930600}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2962930600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.179:6835/994628887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 994628887}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 994628887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.179:6837/164318179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 164318179}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 164318179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.47:6839/1610791508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 1610791508}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 1610791508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.179:6839/2081935059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2081935059}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2081935059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24496, 24485, 24475, 24481, 24455]} 2024-09-19T05:41:18.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T05:41:18.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T05:41:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:19.450 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:19 smithi038 ceph-mon[31227]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 -- 172.21.15.38:0/4146013669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 msgr2=0x7f6e34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 --2- 172.21.15.38:0/4146013669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34100ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009930 tx=0x7f6e2802a420 comp rx=0 tx=0).stop 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 -- 172.21.15.38:0/4146013669 shutdown_connections 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 --2- 172.21.15.38:0/4146013669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34101420 0x7f6e3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 --2- 172.21.15.38:0/4146013669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34100ee0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 --2- 172.21.15.38:0/4146013669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34106a80 0x7f6e34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 -- 172.21.15.38:0/4146013669 >> 172.21.15.38:0/4146013669 conn(0x7f6e340fc7a0 msgr2=0x7f6e340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 -- 172.21.15.38:0/4146013669 shutdown_connections 2024-09-19T05:41:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.530+0000 7f6e390e3640 1 -- 172.21.15.38:0/4146013669 wait complete. 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 Processor -- start 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 -- start start 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34101420 0x7f6e34198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 0x7f6e3419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34110e80 con 0x7f6e34100a80 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34110d00 con 0x7f6e34106a80 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e390e3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34111000 con 0x7f6e34101420 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41162/0 (socket says 172.21.15.38:41162) 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e32d9d640 1 -- 172.21.15.38:0/1582662531 learned_addr learned my addr 172.21.15.38:0/1582662531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34101420 0x7f6e34198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.531+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 0x7f6e3419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 -- 172.21.15.38:0/1582662531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34101420 msgr2=0x7f6e34198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34101420 0x7f6e34198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 -- 172.21.15.38:0/1582662531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 msgr2=0x7f6e3419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 0x7f6e3419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 -- 172.21.15.38:0/1582662531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3419d400 con 0x7f6e34100a80 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 0x7f6e3419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e1bfff640 1 -- 172.21.15.38:0/1582662531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24017070 con 0x7f6e34100a80 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e1bfff640 1 -- 172.21.15.38:0/1582662531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e24009c30 con 0x7f6e34100a80 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3419d6f0 con 0x7f6e34100a80 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e1bfff640 1 -- 172.21.15.38:0/1582662531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400f6b0 con 0x7f6e34100a80 2024-09-19T05:41:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.532+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341a4fd0 con 0x7f6e34100a80 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.534+0000 7f6e1bfff640 1 -- 172.21.15.38:0/1582662531 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6e240109d0 con 0x7f6e34100a80 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.534+0000 7f6e3b4bc740 1 Processor -- start 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.534+0000 7f6e3b4bc740 1 -- start start 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621564db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621571fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5556214cb090 con 0x5556215652f0 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555621472a70 con 0x55562156fbb0 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5556214c7050 con 0x5556215649d0 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e1bfff640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 0x7f6e1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41166/0 (socket says 172.21.15.38:41166) 2024-09-19T05:41:19.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e32d9d640 1 -- 172.21.15.38:0/1417033389 learned_addr learned my addr 172.21.15.38:0/1417033389 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621571fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621564db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 0x7f6e1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e1bfff640 1 -- 172.21.15.38:0/1582662531 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6e240cf8d0 con 0x7f6e34100a80 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 -- 172.21.15.38:0/1417033389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 msgr2=0x555621564db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621564db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 -- 172.21.15.38:0/1417033389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 msgr2=0x55562156f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e3259c640 1 -- 172.21.15.38:0/1417033389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562147f130 con 0x55562156fbb0 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.535+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.536+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 0x7f6e1c0a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009270 tx=0x7f6e2c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.536+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621571fa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e28005940 tx=0x7f6e28005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdc13af35b140f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.536+0000 7f6e197fa640 1 -- 172.21.15.38:0/1417033389 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28040070 con 0x55562156fbb0 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.536+0000 7f6e197fa640 1 -- 172.21.15.38:0/1417033389 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e280319f0 con 0x55562156fbb0 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.536+0000 7f6e197fa640 1 -- 172.21.15.38:0/1417033389 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28035040 con 0x55562156fbb0 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1417033389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 msgr2=0x555621571fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621571fa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e28005940 tx=0x7f6e28005c00 comp rx=0 tx=0).stop 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1417033389 shutdown_connections 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621571fa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1417033389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621564db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1417033389 >> 172.21.15.38:0/1417033389 conn(0x555621560f60 msgr2=0x555621561340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1417033389 shutdown_connections 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1417033389 wait complete. 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- start start 2024-09-19T05:41:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 Processor -- start 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x55562156f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562156c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555621572780 con 0x55562156fbb0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555621485f20 con 0x5556215649d0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555621572900 con 0x5556215652f0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3b4bc740 10 client.0 _ll_register_callbacks cb 0x55562144f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e3359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.537+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x55562156f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41176/0 (socket says 172.21.15.38:41176) 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e32d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x55562156f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:53260/0 (socket says 172.21.15.38:53260) 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 -- 172.21.15.38:0/1885469913 learned_addr learned my addr 172.21.15.38:0/1885469913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562156c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 -- 172.21.15.38:0/1885469913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 msgr2=0x55562156c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562156c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 -- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 msgr2=0x55562156f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x55562156f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 -- 172.21.15.38:0/1885469913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562123c2b0 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009d50 tx=0x7f6e2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85bd5d1e8f9a2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00fd70 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6e2c004b20 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e280319f0 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c004d20 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3b4bc740 10 client.15213 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.538+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55562157df90 con 0x55562156fbb0 2024-09-19T05:41:19.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e280319f0 con 0x55562156fbb0 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6e2c01d040 con 0x55562156fbb0 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 10 client.15213 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 10 client.15213 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 20 client.15213 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x555621580620 0x555621581a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 4 client.15213 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556214d8600 con 0x555621580620 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.539+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x555621580620 0x555621581a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x555621580620 0x555621581a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a92ad947d39e9ce server_cookie=b41b973f2e61bb25 in_seq=0 out_seq=0 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e16ff5640 10 client.15213 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e16ff5640 1 -- 172.21.15.38:0/1885469913 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556214d8600 con 0x555621580620 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e16ff5640 10 client.15213 handle_command_reply: tid=0 2024-09-19T05:41:19.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e3b4bc740 1 client.15213 shutdown 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 msgr2=0x55562156cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009d50 tx=0x7f6e2c004a40 comp rx=0 tx=0).stop 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 shutdown_connections 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.540+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x555621580620 0x555621581a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156cb20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562156c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1885469913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x55562156f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 >> 172.21.15.38:0/1885469913 conn(0x555621560f60 msgr2=0x555621566c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e16ff5640 0 client.15213 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 shutdown_connections 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1885469913 wait complete. 2024-09-19T05:41:19.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.541+0000 7f6e3b4bc740 20 client.15213 trim_cache size 0 max 16384 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 Processor -- start 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 -- start start 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x555621580620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562158a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55562157fdd0 con 0x55562156fbb0 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5556215804a0 con 0x5556215649d0 2024-09-19T05:41:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555621580b70 con 0x5556215652f0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41180/0 (socket says 172.21.15.38:41180) 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3359e640 1 -- 172.21.15.38:0/1695970333 learned_addr learned my addr 172.21.15.38:0/1695970333 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x555621580620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.559+0000 7f6e3259c640 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562158a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 -- 172.21.15.38:0/1695970333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 msgr2=0x55562158a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562158a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 -- 172.21.15.38:0/1695970333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 msgr2=0x555621580620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x555621580620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 -- 172.21.15.38:0/1695970333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621560a00 con 0x55562156fbb0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e3359e640 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c006010 tx=0x7f6e2c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8cadeafbb20d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e18ff9640 1 -- 172.21.15.38:0/1695970333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00f7b0 con 0x55562156fbb0 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e18ff9640 1 -- 172.21.15.38:0/1695970333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c00fb70 con 0x55562156fbb0 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.560+0000 7f6e18ff9640 1 -- 172.21.15.38:0/1695970333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c004ec0 con 0x55562156fbb0 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1695970333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 msgr2=0x55562156c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c006010 tx=0x7f6e2c0098f0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1695970333 shutdown_connections 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55562156fbb0 0x55562156c5a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215652f0 0x55562158a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/1695970333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556215649d0 0x555621580620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1695970333 >> 172.21.15.38:0/1695970333 conn(0x555621560f60 msgr2=0x55562155bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1695970333 shutdown_connections 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/1695970333 wait complete. 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 -- start start 2024-09-19T05:41:19.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.561+0000 7f6e3b4bc740 1 Processor -- start 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621569bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621566b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555621581910 con 0x5556215652f0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555621581240 con 0x55562156fbb0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555621581fe0 con 0x5556215649d0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3b4bc740 10 client.0 _ll_register_callbacks cb 0x7f6e2c014730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41188/0 (socket says 172.21.15.38:41188) 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 -- 172.21.15.38:0/3410256120 learned_addr learned my addr 172.21.15.38:0/3410256120 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3359e640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621566b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621569bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 -- 172.21.15.38:0/3410256120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 msgr2=0x555621569bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621569bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 -- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 msgr2=0x555621566b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621566b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 -- 172.21.15.38:0/3410256120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555621567390 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e3259c640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006010 tx=0x7f6e28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa178c2ad78c5819 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.562+0000 7f6e16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28035050 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6e28004230 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c00fb70 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e28059050 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 10 client.15219 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556215ee880 con 0x5556215652f0 2024-09-19T05:41:19.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c00fb70 con 0x5556215652f0 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6e28004b90 con 0x5556215652f0 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 10 client.15219 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 10 client.15219 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 20 client.15219 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x5556215f3070 0x5556215f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 4 client.15219 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.563+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "status"}) -- 0x5556214d2b10 con 0x5556215f3070 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x5556215f3070 0x5556215f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e32d9d640 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x5556215f3070 0x5556215f1d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7f4355003e31930 server_cookie=3e005fe51d2eebbd in_seq=0 out_seq=0 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e16ff5640 10 client.15219 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e16ff5640 1 -- 172.21.15.38:0/3410256120 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5556214d2b10 con 0x5556215f3070 2024-09-19T05:41:19.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e16ff5640 10 client.15219 handle_command_reply: tid=0 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 client.15219 shutdown 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 msgr2=0x55562156a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006010 tx=0x7f6e28000bc0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 shutdown_connections 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x5556215f3070 0x5556215f1d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55562156fbb0 0x555621566b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5556215652f0 0x55562156a110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 --2- 172.21.15.38:0/3410256120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556215649d0 0x555621569bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 >> 172.21.15.38:0/3410256120 conn(0x555621560f60 msgr2=0x55562156c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.564+0000 7f6e16ff5640 0 client.15219 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 shutdown_connections 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e3b4bc740 1 -- 172.21.15.38:0/3410256120 wait complete. 2024-09-19T05:41:19.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e3b4bc740 20 client.15219 trim_cache size 0 max 16384 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24455, 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 92.467470782000007, 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:19.581 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 159.12554169200001 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 msgr2=0x7f6e1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 0x7f6e1c0a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009270 tx=0x7f6e2c005bd0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 msgr2=0x7f6e34198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:19.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400de10 comp rx=0 tx=0).stop 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 shutdown_connections 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6e1c0a3240 0x7f6e1c0a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e34106a80 0x7f6e3419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34101420 0x7f6e34198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 --2- 172.21.15.38:0/1582662531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e34100a80 0x7f6e34198450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 >> 172.21.15.38:0/1582662531 conn(0x7f6e340fc7a0 msgr2=0x7f6e3406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 shutdown_connections 2024-09-19T05:41:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:19.565+0000 7f6e390e3640 1 -- 172.21.15.38:0/1582662531 wait complete. 2024-09-19T05:41:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:19 smithi179 ceph-mon[35781]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:19 smithi047 ceph-mon[35327]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s; 226 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:20.406 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T05:41:20.667 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 -- 172.21.15.38:0/1222742128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 msgr2=0x7fb70810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 --2- 172.21.15.38:0/1222742128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb70810bb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009900 tx=0x7fb6fc02a480 comp rx=0 tx=0).stop 2024-09-19T05:41:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 -- 172.21.15.38:0/1222742128 shutdown_connections 2024-09-19T05:41:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 --2- 172.21.15.38:0/1222742128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70810c170 0x7fb70810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 --2- 172.21.15.38:0/1222742128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb70810bb50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 --2- 172.21.15.38:0/1222742128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb708102d40 0x7fb708103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 -- 172.21.15.38:0/1222742128 >> 172.21.15.38:0/1222742128 conn(0x7fb7080fc7a0 msgr2=0x7fb7080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 -- 172.21.15.38:0/1222742128 shutdown_connections 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.161+0000 7fb70e162640 1 -- 172.21.15.38:0/1222742128 wait complete. 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 Processor -- start 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 -- start start 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb708102d40 0x7fb708198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70810c170 0x7fb70819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb708111fc0 con 0x7fb708103660 2024-09-19T05:41:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb708111e40 con 0x7fb708102d40 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb70e162640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112140 con 0x7fb70810c170 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41206/0 (socket says 172.21.15.38:41206) 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb706ffd640 1 -- 172.21.15.38:0/2248318407 learned_addr learned my addr 172.21.15.38:0/2248318407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb7077fe640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb708102d40 0x7fb708198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.162+0000 7fb707fff640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70810c170 0x7fb70819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 -- 172.21.15.38:0/2248318407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70810c170 msgr2=0x7fb70819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70810c170 0x7fb70819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 -- 172.21.15.38:0/2248318407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb708102d40 msgr2=0x7fb708198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb708102d40 0x7fb708198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 -- 172.21.15.38:0/2248318407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70819d400 con 0x7fb708103660 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc004770 tx=0x7fb6fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb704ff9640 1 -- 172.21.15.38:0/2248318407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc03f070 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70819d690 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb704ff9640 1 -- 172.21.15.38:0/2248318407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc031ca0 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb704ff9640 1 -- 172.21.15.38:0/2248318407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc035030 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.163+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a4fd0 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.165+0000 7fb704ff9640 1 -- 172.21.15.38:0/2248318407 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb6fc042070 con 0x7fb708103660 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 Processor -- start 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 -- start start 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb704ff9640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 0x7fb6f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e54acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 0x55cb6e559a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e4b1090 con 0x55cb6e559fc0 2024-09-19T05:41:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e458a70 con 0x55cb6e54b1f0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb71053b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e4ad050 con 0x55cb6e54a8d0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb704ff9640 1 -- 172.21.15.38:0/2248318407 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb6fc0f04b0 con 0x7fb708103660 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb707fff640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 0x7fb6f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41214/0 (socket says 172.21.15.38:41214) 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.166+0000 7fb707fff640 1 -- 172.21.15.38:0/2135639292 learned_addr learned my addr 172.21.15.38:0/2135639292 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 0x55cb6e559a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb7077fe640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e54acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 -- 172.21.15.38:0/2135639292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 msgr2=0x55cb6e54acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e54acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 -- 172.21.15.38:0/2135639292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 msgr2=0x55cb6e559a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 0x55cb6e559a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 -- 172.21.15.38:0/2135639292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb6e465130 con 0x55cb6e559fc0 2024-09-19T05:41:21.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 0x55cb6e559a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 0x7fb6f00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb700009d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb707fff640 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb700009280 tx=0x7fb70001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61efc1257f25b20b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb6edffb640 1 -- 172.21.15.38:0/2135639292 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700004b20 con 0x55cb6e559fc0 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb6edffb640 1 -- 172.21.15.38:0/2135639292 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700004cc0 con 0x55cb6e559fc0 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.167+0000 7fb6edffb640 1 -- 172.21.15.38:0/2135639292 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700005430 con 0x55cb6e559fc0 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- 172.21.15.38:0/2135639292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 msgr2=0x55cb6e55c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb700009280 tx=0x7fb70001c000 comp rx=0 tx=0).stop 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- 172.21.15.38:0/2135639292 shutdown_connections 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e55c3b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54b1f0 0x55cb6e559a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 --2- 172.21.15.38:0/2135639292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e54acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- 172.21.15.38:0/2135639292 >> 172.21.15.38:0/2135639292 conn(0x55cb6e546f60 msgr2=0x55cb6e547340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- 172.21.15.38:0/2135639292 shutdown_connections 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- 172.21.15.38:0/2135639292 wait complete. 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.168+0000 7fb71053b740 1 -- start start 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 Processor -- start 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 0x55cb6e559830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 0x55cb6e556b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e5570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e55cb90 con 0x55cb6e559fc0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e46bf20 con 0x55cb6e54a8d0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e55cd10 con 0x55cb6e54b1f0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb71053b740 10 client.0 _ll_register_callbacks cb 0x55cb6e435030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 0x55cb6e559830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 0x55cb6e556b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e5570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 0x55cb6e559830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:53332/0 (socket says 172.21.15.38:53332) 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb7077fe640 1 -- 172.21.15.38:0/1379085034 learned_addr learned my addr 172.21.15.38:0/1379085034 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 0x55cb6e556b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:46558/0 (socket says 172.21.15.38:46558) 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 -- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 msgr2=0x55cb6e556b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 0x55cb6e556b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 -- 172.21.15.38:0/1379085034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 msgr2=0x55cb6e559830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 0x55cb6e559830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.169+0000 7fb707fff640 1 -- 172.21.15.38:0/1379085034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb6e2222b0 con 0x55cb6e559fc0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb707fff640 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e5570c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55cb6e54c470 tx=0x7fb700010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2ca81c0af2ae6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70002e070 con 0x55cb6e559fc0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb70000cac0 con 0x55cb6e559fc0 2024-09-19T05:41:21.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb700004cc0 con 0x55cb6e559fc0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb70003a050 con 0x55cb6e559fc0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb71053b740 10 client.15231 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb6e568510 con 0x55cb6e559fc0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700004cc0 con 0x55cb6e559fc0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb70001f070 con 0x55cb6e559fc0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.170+0000 7fb71053b740 10 client.15231 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb71053b740 10 client.15231 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb71053b740 20 client.15231 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e56af60 0x55cb6e56bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb71053b740 4 client.15231 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb6e4be600 con 0x55cb6e56af60 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e56af60 0x55cb6e56bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb6e6ffd640 10 client.15231 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.171+0000 7fb7077fe640 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e56af60 0x55cb6e56bd00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1fd34aa85273c8c server_cookie=94878a0a4ad3aa6f in_seq=0 out_seq=0 2024-09-19T05:41:21.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/1379085034 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb6e4be600 con 0x55cb6e56af60 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb6e6ffd640 10 client.15231 handle_command_reply: tid=0 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 client.15231 shutdown 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 msgr2=0x55cb6e5570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e5570c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55cb6e54c470 tx=0x7fb700010f40 comp rx=0 tx=0).stop 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 shutdown_connections 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e56af60 0x55cb6e56bd00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e559fc0 0x55cb6e5570c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54b1f0 0x55cb6e556b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb6e6ffd640 0 client.15231 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 --2- 172.21.15.38:0/1379085034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e54a8d0 0x55cb6e559830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.172+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 >> 172.21.15.38:0/1379085034 conn(0x55cb6e546f60 msgr2=0x55cb6e54ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.173+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 shutdown_connections 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.173+0000 7fb71053b740 1 -- 172.21.15.38:0/1379085034 wait complete. 2024-09-19T05:41:21.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.173+0000 7fb71053b740 20 client.15231 trim_cache size 0 max 16384 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.190+0000 7fb71053b740 1 Processor -- start 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.190+0000 7fb71053b740 1 -- start start 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.190+0000 7fb71053b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.190+0000 7fb71053b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.190+0000 7fb71053b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e553de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb71053b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e55cb90 con 0x55cb6e54b1f0 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb71053b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e56a710 con 0x55cb6e559fc0 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb71053b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e56ade0 con 0x55cb6e54a8d0 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41230/0 (socket says 172.21.15.38:41230) 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 -- 172.21.15.38:0/2015859464 learned_addr learned my addr 172.21.15.38:0/2015859464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb707fff640 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e553de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb7077fe640 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 -- 172.21.15.38:0/2015859464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 msgr2=0x55cb6e56bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 -- 172.21.15.38:0/2015859464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 msgr2=0x55cb6e553de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e553de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 -- 172.21.15.38:0/2015859464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb6e54b020 con 0x55cb6e54b1f0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb706ffd640 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0edf60 tx=0x7fb6fc03b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2611e0342717d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb6ed7fa640 1 -- 172.21.15.38:0/2015859464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0d5070 con 0x55cb6e54b1f0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb6ed7fa640 1 -- 172.21.15.38:0/2015859464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0bce10 con 0x55cb6e54b1f0 2024-09-19T05:41:21.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.191+0000 7fb6ed7fa640 1 -- 172.21.15.38:0/2015859464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc049040 con 0x55cb6e54b1f0 2024-09-19T05:41:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- 172.21.15.38:0/2015859464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 msgr2=0x55cb6e569f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0edf60 tx=0x7fb6fc03b890 comp rx=0 tx=0).stop 2024-09-19T05:41:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- 172.21.15.38:0/2015859464 shutdown_connections 2024-09-19T05:41:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e553de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e569f90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 --2- 172.21.15.38:0/2015859464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- 172.21.15.38:0/2015859464 >> 172.21.15.38:0/2015859464 conn(0x55cb6e546f60 msgr2=0x55cb6e552240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- 172.21.15.38:0/2015859464 shutdown_connections 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- 172.21.15.38:0/2015859464 wait complete. 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.192+0000 7fb71053b740 1 -- start start 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 Processor -- start 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e5d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e56bb80 con 0x55cb6e54b1f0 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e56b4b0 con 0x55cb6e559fc0 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cb6e56c250 con 0x55cb6e54a8d0 2024-09-19T05:41:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb71053b740 10 client.0 _ll_register_callbacks cb 0x7fb6fc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41234/0 (socket says 172.21.15.38:41234) 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 -- 172.21.15.38:0/3835441412 learned_addr learned my addr 172.21.15.38:0/3835441412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb7077fe640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb707fff640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e5d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 -- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 msgr2=0x55cb6e56d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 -- 172.21.15.38:0/3835441412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 msgr2=0x55cb6e5d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e5d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb706ffd640 1 -- 172.21.15.38:0/3835441412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb6e5d3550 con 0x55cb6e54b1f0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.193+0000 7fb707fff640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e5d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb706ffd640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc031550 tx=0x7fb6fc005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96caa837caabae68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:21.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc049070 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb6fc0b8d80 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc0bce10 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0d5070 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 10 client.15237 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb6e5d4850 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc0bce10 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb6fc0f0130 con 0x55cb6e54b1f0 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 10 client.15237 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 10 client.15237 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 20 client.15237 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e5d91c0 0x55cb6e5d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:21.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 4 client.15237 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.194+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "status"}) -- 0x55cb6e4b8b10 con 0x55cb6e5d91c0 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb7077fe640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e5d91c0 0x55cb6e5d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb7077fe640 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e5d91c0 0x55cb6e5d7ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f177a42e059cc91e server_cookie=97e237bda9ecbe43 in_seq=0 out_seq=0 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb6e6ffd640 10 client.15237 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb6e6ffd640 1 -- 172.21.15.38:0/3835441412 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cb6e4b8b10 con 0x55cb6e5d91c0 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb6e6ffd640 10 client.15237 handle_command_reply: tid=0 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb71053b740 1 client.15237 shutdown 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 msgr2=0x55cb6e56d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.195+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc031550 tx=0x7fb6fc005e10 comp rx=0 tx=0).stop 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 shutdown_connections 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb6e559fc0 0x55cb6e5d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x55cb6e5d91c0 0x55cb6e5d7ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cb6e54b1f0 0x55cb6e56d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 --2- 172.21.15.38:0/3835441412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cb6e54a8d0 0x55cb6e56d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 >> 172.21.15.38:0/3835441412 conn(0x55cb6e546f60 msgr2=0x55cb6e552240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb6e6ffd640 0 client.15237 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 shutdown_connections 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 1 -- 172.21.15.38:0/3835441412 wait complete. 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb71053b740 20 client.15237 trim_cache size 0 max 16384 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24475, 2024-09-19T05:41:21.204 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 95.105726548000007, 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 165.982541094 2024-09-19T05:41:21.205 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 msgr2=0x7fb6f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 0x7fb6f00a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb700009d00 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 msgr2=0x7fb708198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc004770 tx=0x7fb6fc003c10 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 shutdown_connections 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb6f00a3240 0x7fb6f00a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70810c170 0x7fb70819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb708103660 0x7fb708198990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 --2- 172.21.15.38:0/2248318407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb708102d40 0x7fb708198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.196+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 >> 172.21.15.38:0/2248318407 conn(0x7fb7080fc7a0 msgr2=0x7fb7080fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:21.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.197+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 shutdown_connections 2024-09-19T05:41:21.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:21.197+0000 7fb70e162640 1 -- 172.21.15.38:0/2248318407 wait complete. 2024-09-19T05:41:21.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:21 smithi038 ceph-mon[31227]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 977 B/s wr, 10 op/s; 192 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:21.528 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:21 smithi047 ceph-mon[35327]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 977 B/s wr, 10 op/s; 192 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:21 smithi179 ceph-mon[35781]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 977 B/s wr, 10 op/s; 192 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:41:21.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:41:22.190 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- 172.21.15.38:0/3081689872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd4074620 msgr2=0x7fddd4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/3081689872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd4074620 0x7fddd4074a00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009900 tx=0x7fddd002a480 comp rx=0 tx=0).stop 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- 172.21.15.38:0/3081689872 shutdown_connections 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/3081689872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd410e3c0 0x7fddd4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/3081689872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 0x7fddd410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/3081689872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd4074620 0x7fddd4074a00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- 172.21.15.38:0/3081689872 >> 172.21.15.38:0/3081689872 conn(0x7fddd40fd8c0 msgr2=0x7fddd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- 172.21.15.38:0/3081689872 shutdown_connections 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- 172.21.15.38:0/3081689872 wait complete. 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 Processor -- start 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.693+0000 7fdddbcbf640 1 -- start start 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 0x7fddd419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 0x7fddd419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4114170 con 0x7fddd410e3c0 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4113ff0 con 0x7fddd4074f40 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fdddbcbf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fddd41142f0 con 0x7fddd4074620 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41240/0 (socket says 172.21.15.38:41240) 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 -- 172.21.15.38:0/2847718287 learned_addr learned my addr 172.21.15.38:0/2847718287 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddd925a640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 0x7fddd419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 0x7fddd419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 -- 172.21.15.38:0/2847718287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 msgr2=0x7fddd419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 0x7fddd419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 -- 172.21.15.38:0/2847718287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 msgr2=0x7fddd419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 0x7fddd419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 -- 172.21.15.38:0/2847718287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd419f620 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 0x7fddd419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddda25c640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00cad0 tx=0x7fddcc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.694+0000 7fddcaffd640 1 -- 172.21.15.38:0/2847718287 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00ae20 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.695+0000 7fddcaffd640 1 -- 172.21.15.38:0/2847718287 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddcc005020 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.695+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd419f910 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.695+0000 7fddcaffd640 1 -- 172.21.15.38:0/2847718287 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00f6f0 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.695+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd41a71f0 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.697+0000 7fddcaffd640 1 -- 172.21.15.38:0/2847718287 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fddcc0051d0 con 0x7fddd410e3c0 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.697+0000 7fddde098740 1 Processor -- start 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.697+0000 7fddde098740 1 -- start start 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96cae7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96caf4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca4e090 con 0x55d96cae82f0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d96c9f5a70 con 0x55d96caf2bb0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddde098740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca4a050 con 0x55d96cae79d0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41248/0 (socket says 172.21.15.38:41248) 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 -- 172.21.15.38:0/3992845363 learned_addr learned my addr 172.21.15.38:0/3992845363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd925a640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96cae7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd925a640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96caf4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddcaffd640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 0x7fddc00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 -- 172.21.15.38:0/3992845363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 msgr2=0x55d96cae7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96cae7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 -- 172.21.15.38:0/3992845363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 msgr2=0x55d96caf4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96caf4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 -- 172.21.15.38:0/3992845363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96ca02130 con 0x55d96cae82f0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd925a640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96caf4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddda25c640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 0x7fddc00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddcaffd640 1 -- 172.21.15.38:0/2847718287 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fddcc0cfff0 con 0x7fddd410e3c0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddd0033000 tx=0x7fddd0032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1380bebc84d1eda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.698+0000 7fddbffff640 1 -- 172.21.15.38:0/3992845363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd003f070 con 0x55d96cae82f0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddbffff640 1 -- 172.21.15.38:0/3992845363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd00317b0 con 0x55d96cae82f0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddbffff640 1 -- 172.21.15.38:0/3992845363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0035070 con 0x55d96cae82f0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 -- 172.21.15.38:0/3992845363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 msgr2=0x55d96caf2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddda25c640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 0x7fddc00a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00a6e0 tx=0x7fddcc0bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fddd0033000 tx=0x7fddd0032d60 comp rx=0 tx=0).stop 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 -- 172.21.15.38:0/3992845363 shutdown_connections 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96caf4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae82f0 0x55d96caf2670 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 --2- 172.21.15.38:0/3992845363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96cae7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 -- 172.21.15.38:0/3992845363 >> 172.21.15.38:0/3992845363 conn(0x55d96cae3f60 msgr2=0x55d96cae4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 -- 172.21.15.38:0/3992845363 shutdown_connections 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.699+0000 7fddde098740 1 -- 172.21.15.38:0/3992845363 wait complete. 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 -- start start 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 Processor -- start 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96caf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 0x55d96caef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d96caf5780 con 0x55d96caf2bb0 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d96ca08f20 con 0x55d96cae82f0 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d96caf5900 con 0x55d96cae79d0 2024-09-19T05:41:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddde098740 10 client.0 _ll_register_callbacks cb 0x55d96c9d2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.700+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41258/0 (socket says 172.21.15.38:41258) 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 -- 172.21.15.38:0/2018026518 learned_addr learned my addr 172.21.15.38:0/2018026518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 0x55d96caef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96caf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 -- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 msgr2=0x55d96caf2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96caf2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 -- 172.21.15.38:0/2018026518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 msgr2=0x55d96caef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 0x55d96caef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 -- 172.21.15.38:0/2018026518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96c7bf2b0 con 0x55d96caf2bb0 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 0x55d96caef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddd925a640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddc400c010 tx=0x7fddc400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc8dea251ffedd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4013e60 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fddc400ec60 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddd00317b0 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddc400ee60 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddde098740 10 client.15249 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.701+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d96cb01100 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd00317b0 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fddc4020450 con 0x55d96caf2bb0 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 10 client.15249 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 20 client.15249 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb03700 0x55d96cb04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 4 client.15249 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d96ca5b600 con 0x55d96cb03700 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.702+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb03700 0x55d96cb04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddbd7fa640 10 client.15249 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb03700 0x55d96cb04b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=102035a924c5500b server_cookie=4fe899fc2009cf68 in_seq=0 out_seq=0 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddbd7fa640 1 -- 172.21.15.38:0/2018026518 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d96ca5b600 con 0x55d96cb03700 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddbd7fa640 10 client.15249 handle_command_reply: tid=0 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddde098740 1 client.15249 shutdown 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.703+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 msgr2=0x55d96caefce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fddc400c010 tx=0x7fddc400bbb0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 shutdown_connections 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb03700 0x55d96cb04b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96caefce0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae82f0 0x55d96caef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 --2- 172.21.15.38:0/2018026518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae79d0 0x55d96caf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddbd7fa640 0 client.15249 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 >> 172.21.15.38:0/2018026518 conn(0x55d96cae3f60 msgr2=0x55d96cae9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 shutdown_connections 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 1 -- 172.21.15.38:0/2018026518 wait complete. 2024-09-19T05:41:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.704+0000 7fddde098740 20 client.15249 trim_cache size 0 max 16384 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 Processor -- start 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 -- start start 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae79d0 0x55d96cb04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb0ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb02eb0 con 0x55d96caf2bb0 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb03580 con 0x55d96cae79d0 2024-09-19T05:41:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.722+0000 7fddde098740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb03c50 con 0x55d96cae82f0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41268/0 (socket says 172.21.15.38:41268) 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 -- 172.21.15.38:0/3051697687 learned_addr learned my addr 172.21.15.38:0/3051697687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae79d0 0x55d96cb04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb0ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 -- 172.21.15.38:0/3051697687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 msgr2=0x55d96cb0ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb0ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 -- 172.21.15.38:0/3051697687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae79d0 msgr2=0x55d96cb04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae79d0 0x55d96cb04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 -- 172.21.15.38:0/3051697687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96cae8120 con 0x55d96caf2bb0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddd925a640 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fddc400bcf0 tx=0x7fddc400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96075ce722d8488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddbf7fe640 1 -- 172.21.15.38:0/3051697687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4013e60 con 0x55d96caf2bb0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddbf7fe640 1 -- 172.21.15.38:0/3051697687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc400ea60 con 0x55d96caf2bb0 2024-09-19T05:41:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.723+0000 7fddbf7fe640 1 -- 172.21.15.38:0/3051697687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4003a60 con 0x55d96caf2bb0 2024-09-19T05:41:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- 172.21.15.38:0/3051697687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 msgr2=0x55d96cb0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fddc400bcf0 tx=0x7fddc400bc20 comp rx=0 tx=0).stop 2024-09-19T05:41:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- 172.21.15.38:0/3051697687 shutdown_connections 2024-09-19T05:41:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96caf2bb0 0x55d96cb0d390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb0ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 --2- 172.21.15.38:0/3051697687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96cae79d0 0x55d96cb04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- 172.21.15.38:0/3051697687 >> 172.21.15.38:0/3051697687 conn(0x55d96cae3f60 msgr2=0x55d96cae4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- 172.21.15.38:0/3051697687 shutdown_connections 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- 172.21.15.38:0/3051697687 wait complete. 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.724+0000 7fddde098740 1 -- start start 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 Processor -- start 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb08990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96cb65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb049f0 con 0x55d96cae79d0 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb04320 con 0x55d96caf2bb0 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d96cb050c0 con 0x55d96cae82f0 2024-09-19T05:41:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddde098740 10 client.0 _ll_register_callbacks cb 0x7fddc40136a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41282/0 (socket says 172.21.15.38:41282) 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 -- 172.21.15.38:0/1585348475 learned_addr learned my addr 172.21.15.38:0/1585348475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb08990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 -- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 msgr2=0x55d96cb08990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb08990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 -- 172.21.15.38:0/1585348475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 msgr2=0x55d96cb65e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96cb65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.725+0000 7fddd925a640 1 -- 172.21.15.38:0/1585348475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d96cb66480 con 0x55d96cae79d0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddd925a640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddc400ba50 tx=0x7fddc4013510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9375492ba492b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:22.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4013e60 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fddc402e430 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc400ea60 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddc402e630 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddde098740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d96cb6d7d0 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc400ea60 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fddc40108a0 con 0x55d96cae79d0 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddde098740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.726+0000 7fddde098740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddde098740 20 client.15255 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb71f90 0x55d96cb70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddde098740 4 client.15255 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "status"}) -- 0x55d96ca55b10 con 0x55d96cb71f90 2024-09-19T05:41:22.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb71f90 0x55d96cb70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddbd7fa640 10 client.15255 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.727+0000 7fddd9a5b640 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb71f90 0x55d96cb70ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7b52ab96f3ea376 server_cookie=54944e3e56e9f6ec in_seq=0 out_seq=0 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddbd7fa640 1 -- 172.21.15.38:0/1585348475 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d96ca55b10 con 0x55d96cb71f90 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddbd7fa640 10 client.15255 handle_command_reply: tid=0 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 client.15255 shutdown 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 msgr2=0x55d96cb08450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddc400ba50 tx=0x7fddc4013510 comp rx=0 tx=0).stop 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 shutdown_connections 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55d96cb71f90 0x55d96cb70ca0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d96caf2bb0 0x55d96cb65e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d96cae82f0 0x55d96cb08990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 --2- 172.21.15.38:0/1585348475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d96cae79d0 0x55d96cb08450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddbd7fa640 0 client.15255 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 >> 172.21.15.38:0/1585348475 conn(0x55d96cae3f60 msgr2=0x55d96cae27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 shutdown_connections 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 1 -- 172.21.15.38:0/1585348475 wait complete. 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fddde098740 20 client.15255 trim_cache size 0 max 16384 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24481, 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 97.645466893000005, 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 164.90031335800001 2024-09-19T05:41:22.737 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.728+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 msgr2=0x7fddc00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 0x7fddc00a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00a6e0 tx=0x7fddcc0bc040 comp rx=0 tx=0).stop 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 msgr2=0x7fddd419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00cad0 tx=0x7fddcc00cf90 comp rx=0 tx=0).stop 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 shutdown_connections 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fddc00a3240 0x7fddc00a5700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddd410e3c0 0x7fddd419ef40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddd4074f40 0x7fddd419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 --2- 172.21.15.38:0/2847718287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd4074620 0x7fddd419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 >> 172.21.15.38:0/2847718287 conn(0x7fddd40fd8c0 msgr2=0x7fddd40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 shutdown_connections 2024-09-19T05:41:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:22.729+0000 7fdddbcbf640 1 -- 172.21.15.38:0/2847718287 wait complete. 2024-09-19T05:41:23.486 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:41:23.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:23 smithi038 ceph-mon[31227]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 884 B/s wr, 12 op/s; 174 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:23 smithi179 ceph-mon[35781]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 884 B/s wr, 12 op/s; 174 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:23.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:23 smithi047 ceph-mon[35327]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 884 B/s wr, 12 op/s; 174 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:23.747 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 -- 172.21.15.38:0/84104132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102790 msgr2=0x7f4b701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/84104132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102790 0x7f4b701067e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b6402a410 comp rx=0 tx=0).stop 2024-09-19T05:41:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 -- 172.21.15.38:0/84104132 shutdown_connections 2024-09-19T05:41:24.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/84104132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 0x7f4b7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/84104132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70102790 0x7f4b701067e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/84104132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70101de0 0x7f4b701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 -- 172.21.15.38:0/84104132 >> 172.21.15.38:0/84104132 conn(0x7f4b700fd8e0 msgr2=0x7f4b700ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 -- 172.21.15.38:0/84104132 shutdown_connections 2024-09-19T05:41:24.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.246+0000 7f4b74fb7640 1 -- 172.21.15.38:0/84104132 wait complete. 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 Processor -- start 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 -- start start 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70102790 0x7f4b701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 0x7f4b7019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70111f70 con 0x7f4b70101de0 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70111df0 con 0x7f4b70106f10 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b74fb7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701120f0 con 0x7f4b70102790 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.247+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41292/0 (socket says 172.21.15.38:41292) 2024-09-19T05:41:24.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2718034157 learned_addr learned my addr 172.21.15.38:0/2718034157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 0x7f4b7019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70102790 0x7f4b701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2718034157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70102790 msgr2=0x7f4b701989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70102790 0x7f4b701989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2718034157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 msgr2=0x7f4b7019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 0x7f4b7019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2718034157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7019d460 con 0x7f4b70101de0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e9f0 tx=0x7f4b6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b5f7fe640 1 -- 172.21.15.38:0/2718034157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000cd80 con 0x7f4b70101de0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b5f7fe640 1 -- 172.21.15.38:0/2718034157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b60010910 con 0x7f4b70101de0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7019d750 con 0x7f4b70101de0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b5f7fe640 1 -- 172.21.15.38:0/2718034157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60018990 con 0x7f4b70101de0 2024-09-19T05:41:24.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.248+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b701a5030 con 0x7f4b70101de0 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.251+0000 7f4b77390740 1 Processor -- start 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.251+0000 7f4b77390740 1 -- start start 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.251+0000 7f4b77390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 0x55f51df39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.251+0000 7f4b77390740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.251+0000 7f4b77390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b77390740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f51dea0090 con 0x55f51df3a1d0 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b77390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f51de47a70 con 0x55f51df48fc0 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b77390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f51de9c050 con 0x55f51df398b0 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41298/0 (socket says 172.21.15.38:41298) 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 -- 172.21.15.38:0/902576769 learned_addr learned my addr 172.21.15.38:0/902576769 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 0x55f51df39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 -- 172.21.15.38:0/902576769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 msgr2=0x55f51df39c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 0x55f51df39c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 -- 172.21.15.38:0/902576769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 msgr2=0x55f51df4b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df4b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6e59c640 1 -- 172.21.15.38:0/902576769 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51de54130 con 0x55f51df3a1d0 2024-09-19T05:41:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.252+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 0x55f51df39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.253+0000 7f4b5f7fe640 1 -- 172.21.15.38:0/2718034157 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4b60005040 con 0x7f4b70101de0 2024-09-19T05:41:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.253+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b6001a040 tx=0x7f4b60018f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52ce237d6bfd9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b5cff9640 1 -- 172.21.15.38:0/902576769 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600d5870 con 0x55f51df3a1d0 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b5cff9640 1 -- 172.21.15.38:0/902576769 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b600d5b50 con 0x55f51df3a1d0 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b5cff9640 1 -- 172.21.15.38:0/902576769 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600db830 con 0x55f51df3a1d0 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b5f7fe640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 0x7f4b580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 0x7f4b580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b5f7fe640 1 -- 172.21.15.38:0/2718034157 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4b600d1c50 con 0x7f4b70101de0 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b77390740 1 -- 172.21.15.38:0/902576769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 msgr2=0x55f51df48a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.254+0000 7f4b77390740 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4b6001a040 tx=0x7f4b60018f80 comp rx=0 tx=0).stop 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 -- 172.21.15.38:0/902576769 shutdown_connections 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df4b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df48a80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 --2- 172.21.15.38:0/902576769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df398b0 0x55f51df39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 -- 172.21.15.38:0/902576769 >> 172.21.15.38:0/902576769 conn(0x55f51df35f60 msgr2=0x55f51df36340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 -- 172.21.15.38:0/902576769 shutdown_connections 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 -- 172.21.15.38:0/902576769 wait complete. 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 0x7f4b580a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b68005fd0 tx=0x7f4b68009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 -- start start 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.255+0000 7f4b77390740 1 Processor -- start 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 0x55f51df48890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f51df4bb90 con 0x55f51df3a1d0 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f51de5af20 con 0x55f51df398b0 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f51df4bd10 con 0x55f51df48fc0 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b77390740 10 client.0 _ll_register_callbacks cb 0x55f51de24030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:24.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41308/0 (socket says 172.21.15.38:41308) 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 -- 172.21.15.38:0/2317523795 learned_addr learned my addr 172.21.15.38:0/2317523795 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 0x55f51df48890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 -- 172.21.15.38:0/2317523795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 msgr2=0x55f51df46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 -- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 msgr2=0x55f51df48890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 0x55f51df48890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 -- 172.21.15.38:0/2317523795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51dc112b0 con 0x55f51df3a1d0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 0x55f51df48890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402a930 tx=0x7f4b64031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac9eb90b1e9b875 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.256+0000 7f4b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64035040 con 0x55f51df3a1d0 2024-09-19T05:41:24.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b600d5b50 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4b64004240 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b64004440 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 10 client.15267 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f51df57570 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b600d5b50 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f4b6403d6f0 con 0x55f51df3a1d0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 20 client.15267 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51df5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 4 client.15267 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.257+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f51dead600 con 0x55f51df59bd0 2024-09-19T05:41:24.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.258+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51df5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.258+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51df5b040 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e45130cd8b8f286 server_cookie=f2ac9295f4e8ec6b in_seq=0 out_seq=0 2024-09-19T05:41:24.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.258+0000 7f4b55ffb640 10 client.15267 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b55ffb640 1 -- 172.21.15.38:0/2317523795 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f51dead600 con 0x55f51df59bd0 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b55ffb640 10 client.15267 handle_command_reply: tid=0 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 client.15267 shutdown 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 msgr2=0x55f51df45be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402a930 tx=0x7f4b64031850 comp rx=0 tx=0).stop 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 shutdown_connections 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51df5b040 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df3a1d0 0x55f51df45be0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b55ffb640 0 client.15267 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 --2- 172.21.15.38:0/2317523795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df398b0 0x55f51df48890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 >> 172.21.15.38:0/2317523795 conn(0x55f51df35f60 msgr2=0x55f51df3bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 shutdown_connections 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 1 -- 172.21.15.38:0/2317523795 wait complete. 2024-09-19T05:41:24.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.259+0000 7f4b77390740 20 client.15267 trim_cache size 0 max 16384 2024-09-19T05:41:24.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 Processor -- start 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 -- start start 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df3a1d0 0x55f51df454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f51df59380 con 0x55f51df398b0 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f51df59a50 con 0x55f51df48fc0 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b77390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f51df5a120 con 0x55f51df3a1d0 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41316/0 (socket says 172.21.15.38:41316) 2024-09-19T05:41:24.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.277+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2753072323 learned_addr learned my addr 172.21.15.38:0/2753072323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df3a1d0 0x55f51df454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2753072323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df3a1d0 msgr2=0x55f51df454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df3a1d0 0x55f51df454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2753072323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 msgr2=0x55f51df459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 -- 172.21.15.38:0/2753072323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51df3a000 con 0x55f51df398b0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e9c0 tx=0x7f4b60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca4f1b02f80bca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b57fff640 1 -- 172.21.15.38:0/2753072323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60026040 con 0x55f51df398b0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b57fff640 1 -- 172.21.15.38:0/2753072323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6009e100 con 0x55f51df398b0 2024-09-19T05:41:24.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.278+0000 7f4b57fff640 1 -- 172.21.15.38:0/2753072323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6009a770 con 0x55f51df398b0 2024-09-19T05:41:24.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- 172.21.15.38:0/2753072323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 msgr2=0x55f51df5b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e9c0 tx=0x7f4b60000bc0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- 172.21.15.38:0/2753072323 shutdown_connections 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df48fc0 0x55f51df459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df3a1d0 0x55f51df454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 --2- 172.21.15.38:0/2753072323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df5b040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- 172.21.15.38:0/2753072323 >> 172.21.15.38:0/2753072323 conn(0x55f51df35f60 msgr2=0x55f51df41240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- 172.21.15.38:0/2753072323 shutdown_connections 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- 172.21.15.38:0/2753072323 wait complete. 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 -- start start 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.279+0000 7f4b77390740 1 Processor -- start 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df3a1d0 0x55f51df63ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df5c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f51df5aec0 con 0x55f51df398b0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f51df5a7f0 con 0x55f51df3a1d0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f51df5b590 con 0x55f51df48fc0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b77390740 10 client.0 _ll_register_callbacks cb 0x7f4b64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41322/0 (socket says 172.21.15.38:41322) 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 -- 172.21.15.38:0/1836267639 learned_addr learned my addr 172.21.15.38:0/1836267639 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df3a1d0 0x55f51df63ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6ed9d640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df5c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 -- 172.21.15.38:0/1836267639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 msgr2=0x55f51df5c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df5c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 -- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df3a1d0 msgr2=0x55f51df63ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df3a1d0 0x55f51df63ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 -- 172.21.15.38:0/1836267639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51df5d0d0 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b6e59c640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b6009de00 tx=0x7f4b60005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd30a1f1d2f11954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60026070 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.280+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4b600a2110 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b6009e100 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b600a6c30 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f51df61bb0 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b6009e100 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f4b600d9040 con 0x55f51df398b0 2024-09-19T05:41:24.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 20 client.15273 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51dfc71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 4 client.15273 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.281+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "status"}) -- 0x55f51dea7b10 con 0x55f51df59bd0 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51dfc71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b6dd9b640 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51dfc71c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e45d2fb7b260fe7 server_cookie=c862149b9ce8e57b in_seq=0 out_seq=0 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b55ffb640 10 client.15273 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b55ffb640 1 -- 172.21.15.38:0/1836267639 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f51dea7b10 con 0x55f51df59bd0 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b55ffb640 10 client.15273 handle_command_reply: tid=0 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 client.15273 shutdown 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 msgr2=0x55f51df63570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b6009de00 tx=0x7f4b60005ab0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 shutdown_connections 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f51df48fc0 0x55f51df5c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f51df3a1d0 0x55f51df63ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x55f51df59bd0 0x55f51dfc71c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 --2- 172.21.15.38:0/1836267639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f51df398b0 0x55f51df63570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 >> 172.21.15.38:0/1836267639 conn(0x55f51df35f60 msgr2=0x55f51df44380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.282+0000 7f4b55ffb640 0 client.15273 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 shutdown_connections 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b77390740 1 -- 172.21.15.38:0/1836267639 wait complete. 2024-09-19T05:41:24.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b77390740 20 client.15273 trim_cache size 0 max 16384 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24485, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 100.210855827, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:24.292 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 152.42866743600001 2024-09-19T05:41:24.293 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:24.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 msgr2=0x7f4b580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 0x7f4b580a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b68005fd0 tx=0x7f4b68009500 comp rx=0 tx=0).stop 2024-09-19T05:41:24.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 msgr2=0x7f4b701984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:24.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e9f0 tx=0x7f4b6000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 shutdown_connections 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b580a32c0 0x7f4b580a5780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70106f10 0x7f4b7019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b70102790 0x7f4b701989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 --2- 172.21.15.38:0/2718034157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b70101de0 0x7f4b701984b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 >> 172.21.15.38:0/2718034157 conn(0x7f4b700fd8e0 msgr2=0x7f4b700fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 shutdown_connections 2024-09-19T05:41:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:24.283+0000 7f4b74fb7640 1 -- 172.21.15.38:0/2718034157 wait complete. 2024-09-19T05:41:25.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:41:25.349 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:25.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:25 smithi038 ceph-mon[31227]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s; 151 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:25 smithi179 ceph-mon[35781]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s; 151 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:25.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:25 smithi047 ceph-mon[35327]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s; 151 B/s, 0 keys/s, 1 objects/s recovering 2024-09-19T05:41:25.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 -- 172.21.15.38:0/2859466171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac100a80 msgr2=0x7f67ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 --2- 172.21.15.38:0/2859466171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac100a80 0x7f67ac100ee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f679c009cd0 tx=0x7f679c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 -- 172.21.15.38:0/2859466171 shutdown_connections 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 --2- 172.21.15.38:0/2859466171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac101420 0x7f67ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 --2- 172.21.15.38:0/2859466171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac100a80 0x7f67ac100ee0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 --2- 172.21.15.38:0/2859466171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac106a80 0x7f67ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 -- 172.21.15.38:0/2859466171 >> 172.21.15.38:0/2859466171 conn(0x7f67ac0fc7a0 msgr2=0x7f67ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 -- 172.21.15.38:0/2859466171 shutdown_connections 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.824+0000 7f67b32dd640 1 -- 172.21.15.38:0/2859466171 wait complete. 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 Processor -- start 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 -- start start 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac100a80 0x7f67ac199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 0x7f67ac1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac110e80 con 0x7f67ac101420 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac110d00 con 0x7f67ac100a80 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b32dd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac111000 con 0x7f67ac106a80 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41342/0 (socket says 172.21.15.38:41342) 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 -- 172.21.15.38:0/1907394461 learned_addr learned my addr 172.21.15.38:0/1907394461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b187a640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 0x7f67ac1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b1079640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac100a80 0x7f67ac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 -- 172.21.15.38:0/1907394461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 msgr2=0x7f67ac1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 0x7f67ac1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 -- 172.21.15.38:0/1907394461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac100a80 msgr2=0x7f67ac199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac100a80 0x7f67ac199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b0878640 1 -- 172.21.15.38:0/1907394461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac1a47d0 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.825+0000 7f67b187a640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 0x7f67ac1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67b0878640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f679c02ae00 tx=0x7f679c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67a27fc640 1 -- 172.21.15.38:0/1907394461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c004240 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67a27fc640 1 -- 172.21.15.38:0/1907394461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679c0305a0 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac1a49a0 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67a27fc640 1 -- 172.21.15.38:0/1907394461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c03a660 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.826+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a4e50 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.828+0000 7f67a27fc640 1 -- 172.21.15.38:0/1907394461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f679c030ac0 con 0x7f67ac101420 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 Processor -- start 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 -- start start 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b58ba6090 con 0x559b58c4efc0 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559b58b4da70 con 0x559b58c3f890 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b56b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559b58ba2050 con 0x559b58c401b0 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b0878640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b0878640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41352/0 (socket says 172.21.15.38:41352) 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b0878640 1 -- 172.21.15.38:0/1985529524 learned_addr learned my addr 172.21.15.38:0/1985529524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b1079640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67b187a640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.829+0000 7f67a27fc640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 0x7f67940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 -- 172.21.15.38:0/1985529524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 msgr2=0x559b58c4ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c4ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 -- 172.21.15.38:0/1985529524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 msgr2=0x559b58c3fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c3fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67a27fc640 1 -- 172.21.15.38:0/1907394461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f679c02e030 con 0x7f67ac101420 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 -- 172.21.15.38:0/1985529524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b58b5a130 con 0x559b58c4efc0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b1079640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c4ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b187a640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c3fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b187a640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 0x7f67940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67b0878640 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f679c0ee4e0 tx=0x7f679c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd33fc4fcb27b4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67937fe640 1 -- 172.21.15.38:0/1985529524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0bca20 con 0x559b58c4efc0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67937fe640 1 -- 172.21.15.38:0/1985529524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679c036020 con 0x559b58c4efc0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.830+0000 7f67937fe640 1 -- 172.21.15.38:0/1985529524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0c1470 con 0x559b58c4efc0 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- 172.21.15.38:0/1985529524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 msgr2=0x559b58c513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f679c0ee4e0 tx=0x7f679c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b187a640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 0x7f67940a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cd50 tx=0x7f67a40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- 172.21.15.38:0/1985529524 shutdown_connections 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c513b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c4ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 --2- 172.21.15.38:0/1985529524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c3fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- 172.21.15.38:0/1985529524 >> 172.21.15.38:0/1985529524 conn(0x559b58c3bf60 msgr2=0x559b58c3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- 172.21.15.38:0/1985529524 shutdown_connections 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- 172.21.15.38:0/1985529524 wait complete. 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 -- start start 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.831+0000 7f67b56b6740 1 Processor -- start 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c4bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c4efc0 0x559b58c4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b58ba6090 con 0x559b58c3f890 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559b58b60f20 con 0x559b58c401b0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559b58c51950 con 0x559b58c4efc0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b56b6740 10 client.0 _ll_register_callbacks cb 0x559b58b2a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41354/0 (socket says 172.21.15.38:41354) 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 -- 172.21.15.38:0/3098722263 learned_addr learned my addr 172.21.15.38:0/3098722263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b0878640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c4bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b187a640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c4efc0 0x559b58c4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 -- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c4efc0 msgr2=0x559b58c4c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c4efc0 0x559b58c4c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 -- 172.21.15.38:0/3098722263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 msgr2=0x559b58c4bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c4bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 -- 172.21.15.38:0/3098722263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b589172b0 con 0x559b58c3f890 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f67b1079640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f67a800ce80 tx=0x7f67a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912d82dc4152464 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.832+0000 7f6790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8011a50 con 0x559b58c3f890 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679c036020 con 0x559b58c3f890 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f67a8011bf0 con 0x559b58c3f890 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67a8011df0 con 0x559b58c3f890 2024-09-19T05:41:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b58c5d210 con 0x559b58c3f890 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679c036020 con 0x559b58c3f890 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f67a8018070 con 0x559b58c3f890 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 20 client.15285 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58c5f850 0x559b58c60cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 4 client.15285 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.833+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b58bb3600 con 0x559b58c5f850 2024-09-19T05:41:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.834+0000 7f67b0878640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58c5f850 0x559b58c60cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.834+0000 7f67b0878640 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58c5f850 0x559b58c60cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10efe036a5130647 server_cookie=6e8b060718470ca8 in_seq=0 out_seq=0 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.834+0000 7f6790ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f6790ff9640 1 -- 172.21.15.38:0/3098722263 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b58bb3600 con 0x559b58c5f850 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f6790ff9640 10 client.15285 handle_command_reply: tid=0 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 client.15285 shutdown 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 msgr2=0x559b58c4e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f67a800ce80 tx=0x7f67a80039c0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 shutdown_connections 2024-09-19T05:41:25.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58c5f850 0x559b58c60cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c4efc0 0x559b58c4c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c4bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3098722263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c3f890 0x559b58c4e860 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f6790ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 >> 172.21.15.38:0/3098722263 conn(0x559b58c3bf60 msgr2=0x559b58c41cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 shutdown_connections 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 1 -- 172.21.15.38:0/3098722263 wait complete. 2024-09-19T05:41:25.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.835+0000 7f67b56b6740 20 client.15285 trim_cache size 0 max 16384 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.853+0000 7f67b56b6740 1 Processor -- start 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.853+0000 7f67b56b6740 1 -- start start 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c3f890 0x559b58c48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b58c5f000 con 0x559b58c4efc0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559b58c5f6d0 con 0x559b58c401b0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b56b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559b58c5fda0 con 0x559b58c3f890 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41364/0 (socket says 172.21.15.38:41364) 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 -- 172.21.15.38:0/2896016959 learned_addr learned my addr 172.21.15.38:0/2896016959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b1079640 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c3f890 0x559b58c48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b0878640 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 -- 172.21.15.38:0/2896016959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c3f890 msgr2=0x559b58c48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c3f890 0x559b58c48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 -- 172.21.15.38:0/2896016959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 msgr2=0x559b58c49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 -- 172.21.15.38:0/2896016959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b58c3bd90 con 0x559b58c4efc0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f67b187a640 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67a4012580 tx=0x7f67a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4491ff0107b28639 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.854+0000 7f6792ffd640 1 -- 172.21.15.38:0/2896016959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004300 con 0x559b58c4efc0 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f6792ffd640 1 -- 172.21.15.38:0/2896016959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40044a0 con 0x559b58c4efc0 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f6792ffd640 1 -- 172.21.15.38:0/2896016959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4003e20 con 0x559b58c4efc0 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- 172.21.15.38:0/2896016959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 msgr2=0x559b58c46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67a4012580 tx=0x7f67a4013000 comp rx=0 tx=0).stop 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- 172.21.15.38:0/2896016959 shutdown_connections 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58c46a20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c401b0 0x559b58c49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 --2- 172.21.15.38:0/2896016959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c3f890 0x559b58c48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- 172.21.15.38:0/2896016959 >> 172.21.15.38:0/2896016959 conn(0x559b58c3bf60 msgr2=0x559b58c4a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- 172.21.15.38:0/2896016959 shutdown_connections 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- 172.21.15.38:0/2896016959 wait complete. 2024-09-19T05:41:25.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.855+0000 7f67b56b6740 1 -- start start 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 Processor -- start 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c62520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x559b58c60b40 con 0x559b58c4efc0 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559b58c60470 con 0x559b58c3f890 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559b58c61210 con 0x559b58c401b0 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b56b6740 10 client.0 _ll_register_callbacks cb 0x7f67a80041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b187a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b187a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41368/0 (socket says 172.21.15.38:41368) 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b1079640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c62520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b187a640 1 -- 172.21.15.38:0/3482859744 learned_addr learned my addr 172.21.15.38:0/3482859744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:25.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.856+0000 7f67b0878640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 -- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 msgr2=0x559b58c62a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c62a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 -- 172.21.15.38:0/3482859744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 msgr2=0x559b58c62520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c62520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 -- 172.21.15.38:0/3482859744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b58cc84e0 con 0x559b58c4efc0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b1079640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c62520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b187a640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67a4003920 tx=0x7f67a4003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d642644a1bcc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f6790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4012a70 con 0x559b58c4efc0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f67a4022040 con 0x559b58c4efc0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b58badb10 con 0x559b58c4efc0 2024-09-19T05:41:25.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67a4012c10 con 0x559b58c4efc0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b56b6740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b58cc97b0 con 0x559b58c4efc0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.857+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559b58badb10 con 0x559b58c4efc0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f67a401c420 con 0x559b58c4efc0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 20 client.15291 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58cce220 0x559b58cccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 4 client.15291 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "status"}) -- 0x559b58b2a0e0 con 0x559b58cce220 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b1079640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58cce220 0x559b58cccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f6790ff9640 10 client.15291 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.858+0000 7f67b1079640 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58cce220 0x559b58cccf30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2607fcc315da38 server_cookie=1037e49d4a250051 in_seq=0 out_seq=0 2024-09-19T05:41:25.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f6790ff9640 1 -- 172.21.15.38:0/3482859744 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559b58b2a0e0 con 0x559b58cce220 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f6790ff9640 10 client.15291 handle_command_reply: tid=0 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 client.15291 shutdown 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 msgr2=0x559b58cc7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67a4003920 tx=0x7f67a4003fc0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 shutdown_connections 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x559b58cce220 0x559b58cccf30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b58c4efc0 0x559b58cc7f50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559b58c401b0 0x559b58c62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 --2- 172.21.15.38:0/3482859744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b58c3f890 0x559b58c62520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f6790ff9640 0 client.15291 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 >> 172.21.15.38:0/3482859744 conn(0x559b58c3bf60 msgr2=0x559b58c46a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 shutdown_connections 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 1 -- 172.21.15.38:0/3482859744 wait complete. 2024-09-19T05:41:25.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.859+0000 7f67b56b6740 20 client.15291 trim_cache size 0 max 16384 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24496, 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 142.95811429899999, 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 27, 2024-09-19T05:41:25.875 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:25.876 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 87, 2024-09-19T05:41:25.876 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 162.8168666 2024-09-19T05:41:25.876 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 msgr2=0x7f67940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 0x7f67940a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cd50 tx=0x7f67a40046c0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 msgr2=0x7f67ac199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f679c02ae00 tx=0x7f679c003ba0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 shutdown_connections 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f67940a3240 0x7f67940a5700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67ac106a80 0x7f67ac1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67ac101420 0x7f67ac199750 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 --2- 172.21.15.38:0/1907394461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ac100a80 0x7f67ac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 >> 172.21.15.38:0/1907394461 conn(0x7f67ac0fc7a0 msgr2=0x7f67ac0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 shutdown_connections 2024-09-19T05:41:25.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:25.860+0000 7f67b32dd640 1 -- 172.21.15.38:0/1907394461 wait complete. 2024-09-19T05:41:26.659 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T05:41:26.678 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T05:41:26.678 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T05:41:26.679 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T05:41:26.679 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T05:41:26.751 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T05:41:26.751 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T05:41:26.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:26.751 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-19T05:41:26.772 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-19T05:41:26.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:26.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T05:41:26.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:26.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-19T05:41:26.960 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T05:41:26.961 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T05:41:26.961 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:26.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T05:41:27.224 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:27.517 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:27 smithi038 ceph-mon[31227]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:41:27.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:27 smithi179 ceph-mon[35781]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:41:27.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:27 smithi047 ceph-mon[35327]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:41:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 -- 172.21.15.38:0/3879726948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 msgr2=0x7f70301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 --2- 172.21.15.38:0/3879726948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70301021c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7020009900 tx=0x7f702002a410 comp rx=0 tx=0).stop 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 -- 172.21.15.38:0/3879726948 shutdown_connections 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 --2- 172.21.15.38:0/3879726948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 0x7f703010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 --2- 172.21.15.38:0/3879726948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f70301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 --2- 172.21.15.38:0/3879726948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70301021c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 -- 172.21.15.38:0/3879726948 >> 172.21.15.38:0/3879726948 conn(0x7f70300fd8c0 msgr2=0x7f70300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 -- 172.21.15.38:0/3879726948 shutdown_connections 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.721+0000 7f7035c59640 1 -- 172.21.15.38:0/3879726948 wait complete. 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 Processor -- start 2024-09-19T05:41:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 -- start start 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f7030074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 0x7f7030074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7030111f70 con 0x7f7030101de0 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7030111df0 con 0x7f7030102790 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f7035c59640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f70301120f0 con 0x7f7030106f10 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59732/0 (socket says 172.21.15.38:59732) 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f702f7fe640 1 -- 172.21.15.38:0/656006775 learned_addr learned my addr 172.21.15.38:0/656006775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:27.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.722+0000 7f702ffff640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 0x7f7030074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702effd640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f7030074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 -- 172.21.15.38:0/656006775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 msgr2=0x7f7030074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 0x7f7030074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 -- 172.21.15.38:0/656006775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 msgr2=0x7f7030074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f7030074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 -- 172.21.15.38:0/656006775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70301a68f0 con 0x7f7030101de0 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702effd640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f7030074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:27.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702f7fe640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f70200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702003f030 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301a6b80 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7020003ce0 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020035070 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.723+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301a7060 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.724+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70301021c0 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.725+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7020042030 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.726+0000 7f702cff9640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 0x7f70180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.726+0000 7f702effd640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 0x7f70180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.726+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f70200f0290 con 0x7f7030101de0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.726+0000 7f702effd640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 0x7f70180a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7030075f80 tx=0x7f7024005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:27.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.730+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70200bccb0 con 0x7f7030101de0 2024-09-19T05:41:27.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.941+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7030109b60 con 0x7f7030101de0 2024-09-19T05:41:27.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.941+0000 7f702cff9640 1 -- 172.21.15.38:0/656006775 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f70200bc5b0 con 0x7f7030101de0 2024-09-19T05:41:27.943 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T05:41:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 msgr2=0x7f70180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 0x7f70180a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7030075f80 tx=0x7f7024005e50 comp rx=0 tx=0).stop 2024-09-19T05:41:27.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 msgr2=0x7f70300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f70200042f0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 shutdown_connections 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f70180a3240 0x7f70180a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7030106f10 0x7f7030074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7030102790 0x7f7030074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 --2- 172.21.15.38:0/656006775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7030101de0 0x7f70300744f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.943+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 >> 172.21.15.38:0/656006775 conn(0x7f70300fd8c0 msgr2=0x7f70300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.944+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 shutdown_connections 2024-09-19T05:41:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:27.944+0000 7f7035c59640 1 -- 172.21.15.38:0/656006775 wait complete. 2024-09-19T05:41:28.628 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:28 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/656006775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T05:41:28.660 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:41:28.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:28.661 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-19T05:41:28.679 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-19T05:41:28.680 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-19T05:41:28.681 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-19T05:41:28.681 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T05:41:28.682 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:41:28.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:28 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/656006775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:41:28.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:28 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/656006775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:41:28.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:28.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:28.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:28.826 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T05:41:28.853 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-19T05:41:28.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:28.854 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-19T05:41:28.910 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-19T05:41:28.910 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T05:41:28.910 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T05:41:28.910 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T05:41:28.910 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T05:41:28.911 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:41:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:29.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:29.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:29.040 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-19T05:41:29.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:29.058 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-19T05:41:29.113 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:41:29.114 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:41:29.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:41:29.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T05:41:29.114 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:41:29.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:29.232 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T05:41:29.232 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:41:29.232 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:41:29.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T05:41:29.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T05:41:29.232 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T05:41:29.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T05:41:29.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T05:41:29.233 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:41:29.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:29.350 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:41:29.350 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-19T05:41:29.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-19T05:41:29.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T05:41:29.351 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-19T05:41:29.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:29.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:29.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:41:29.430 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:29.446 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:41:29.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:41:29.447 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:29.502 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T05:41:29.502 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T05:41:29.502 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/216dcc11-78f6-472d-8036-e412a733f802 2024-09-19T05:41:29.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:29.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/216dcc11-78f6-472d-8036-e412a733f802 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T05:41:29.564 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:29 smithi038 ceph-mon[31227]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:29.587 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T05:41:29.587 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T05:41:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:29 smithi179 ceph-mon[35781]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:29.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:29 smithi047 ceph-mon[35327]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:41:29.738 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T05:41:29.738 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T05:41:29.738 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.47:3300,172.21.15.179:3300" 2024-09-19T05:41:29.738 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with new device syntax: 0@44ff0e58-7648-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/216dcc11-78f6-472d-8036-e412a733f802 2024-09-19T05:41:29.738 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.38:3300/172.21.15.47:3300/172.21.15.179:3300" will pass to kernel 2024-09-19T05:41:29.739 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T05:41:29.739 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-19T05:41:29.739 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T05:41:29.739 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T05:41:29.739 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T05:41:29.740 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T05:41:29.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:41:29.741 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:29.811 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:41:29.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:41:29.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:29.910 INFO:teuthology.orchestra.run.smithi038.stdout:client25087 2024-09-19T05:41:29.911 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:41:29.911 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:41:29.928 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:41:30.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:41:30.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:30.025 INFO:teuthology.orchestra.run.smithi038.stdout:client25087 2024-09-19T05:41:30.025 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:41:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:41:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:30.082 INFO:teuthology.orchestra.run.smithi038.stdout:client25087 2024-09-19T05:41:30.083 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/44ff0e58-7648-11ef-bceb-c7b262605968.client25087/status') 2024-09-19T05:41:30.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:41:30.143 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.25087 (3)172.21.15.38:0/2531809945 2024-09-19T05:41:30.143 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-09-19T05:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-09-19T05:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-09-19T05:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:71 bytes copied, 2.7198e-05 s, 2.6 MB/s 2024-09-19T05:41:30.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:41:30.151 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T05:41:30.160 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:41:30.161 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:41:30.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 -- 172.21.15.38:0/1334467986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc102eb0 msgr2=0x7f38cc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/1334467986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc102eb0 0x7f38cc103330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009900 tx=0x7f38b802a410 comp rx=0 tx=0).stop 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 -- 172.21.15.38:0/1334467986 shutdown_connections 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/1334467986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc103870 0x7f38cc10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/1334467986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc102eb0 0x7f38cc103330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/1334467986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 0x7f38cc100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 -- 172.21.15.38:0/1334467986 >> 172.21.15.38:0/1334467986 conn(0x7f38cc0fb980 msgr2=0x7f38cc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 -- 172.21.15.38:0/1334467986 shutdown_connections 2024-09-19T05:41:30.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 -- 172.21.15.38:0/1334467986 wait complete. 2024-09-19T05:41:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.265+0000 7f38d2f8d640 1 Processor -- start 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 -- start start 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 0x7f38cc102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc102eb0 0x7f38cc100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d7d0 con 0x7f38cc103870 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d650 con 0x7f38cc102eb0 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d2f8d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d950 con 0x7f38cc0ffe40 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59750/0 (socket says 172.21.15.38:59750) 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 -- 172.21.15.38:0/2644088895 learned_addr learned my addr 172.21.15.38:0/2644088895 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d0d29640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 0x7f38cc102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38cbfff640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc102eb0 0x7f38cc100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 -- 172.21.15.38:0/2644088895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 msgr2=0x7f38cc102820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 0x7f38cc102820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 -- 172.21.15.38:0/2644088895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc102eb0 msgr2=0x7f38cc100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc102eb0 0x7f38cc100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 -- 172.21.15.38:0/2644088895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc101c30 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38d152a640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eb10 tx=0x7f38c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.266+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004b20 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.267+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c4004cc0 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.267+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc1a1ef0 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.267+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4010760 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.267+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc1a2350 con 0x7f38cc103870 2024-09-19T05:41:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.268+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38cc108fe0 con 0x7f38cc103870 2024-09-19T05:41:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.269+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f38c4005020 con 0x7f38cc103870 2024-09-19T05:41:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.270+0000 7f38c9ffb640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 0x7f38bc0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.270+0000 7f38d0d29640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 0x7f38bc0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.270+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f38c40d0130 con 0x7f38cc103870 2024-09-19T05:41:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.270+0000 7f38d0d29640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 0x7f38bc0a5830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f38c0005fd0 tx=0x7f38c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:30.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.274+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38c409ca90 con 0x7f38cc103870 2024-09-19T05:41:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:30.490+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f38cc062e90 con 0x7f38cc103870 2024-09-19T05:41:31.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.289+0000 7f38c9ffb640 1 -- 172.21.15.38:0/2644088895 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f38c409c390 con 0x7f38cc103870 2024-09-19T05:41:31.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.292+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 msgr2=0x7f38bc0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:31.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.292+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 0x7f38bc0a5830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f38c0005fd0 tx=0x7f38c000b410 comp rx=0 tx=0).stop 2024-09-19T05:41:31.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.292+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 msgr2=0x7f38cc1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.292+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eb10 tx=0x7f38c400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.292+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 shutdown_connections 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f38bc0a3370 0x7f38bc0a5830 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38cc103870 0x7f38cc1013a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc102eb0 0x7f38cc100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 --2- 172.21.15.38:0/2644088895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc0ffe40 0x7f38cc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 >> 172.21.15.38:0/2644088895 conn(0x7f38cc0fb980 msgr2=0x7f38cc1061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 shutdown_connections 2024-09-19T05:41:31.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:31.293+0000 7f38d2f8d640 1 -- 172.21.15.38:0/2644088895 wait complete. 2024-09-19T05:41:31.308 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T05:41:31.319 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T05:41:31.363 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:41:31.363 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:41:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:31.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:31 smithi038 ceph-mon[31227]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:31.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T05:41:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:31 smithi179 ceph-mon[35781]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:31 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T05:41:31.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:31 smithi047 ceph-mon[35327]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:41:31.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:31 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 -- 172.21.15.38:0/2068527442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8106f20 msgr2=0x7f4eb810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/2068527442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8106f20 0x7f4eb810eaa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009ca0 tx=0x7f4eb002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 -- 172.21.15.38:0/2068527442 shutdown_connections 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/2068527442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8106f20 0x7f4eb810eaa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/2068527442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 0x7f4eb81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/2068527442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8101df0 0x7f4eb81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 -- 172.21.15.38:0/2068527442 >> 172.21.15.38:0/2068527442 conn(0x7f4eb80fd8f0 msgr2=0x7f4eb80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 -- 172.21.15.38:0/2068527442 shutdown_connections 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.143+0000 7f4ebfed0640 1 -- 172.21.15.38:0/2068527442 wait complete. 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 Processor -- start 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 -- start start 2024-09-19T05:41:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 0x7f4eb8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 0x7f4eb8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8111fd0 con 0x7f4eb8101df0 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8111e50 con 0x7f4eb81027a0 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebfed0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8112150 con 0x7f4eb8106f20 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebdc6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebdc6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59766/0 (socket says 172.21.15.38:59766) 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebdc6c640 1 -- 172.21.15.38:0/4278733723 learned_addr learned my addr 172.21.15.38:0/4278733723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.144+0000 7f4ebd46b640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 0x7f4eb8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebe46d640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 0x7f4eb8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 -- 172.21.15.38:0/4278733723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 msgr2=0x7f4eb8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 0x7f4eb8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 -- 172.21.15.38:0/4278733723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 msgr2=0x7f4eb8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 0x7f4eb8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 -- 172.21.15.38:0/4278733723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a6950 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebe46d640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 0x7f4eb8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebdc6c640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c970 tx=0x7f4eb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007bb0 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb4007d50 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a6c40 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400f450 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.145+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a7130 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.146+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb81021d0 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.147+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4eb4005020 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.148+0000 7f4eaeffd640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 0x7f4ea40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.148+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4eb40cfb70 con 0x7f4eb8101df0 2024-09-19T05:41:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.148+0000 7f4ebd46b640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 0x7f4ea40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.148+0000 7f4ebd46b640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 0x7f4ea40a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005e00 tx=0x7f4ea8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.152+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4eb4014030 con 0x7f4eb8101df0 2024-09-19T05:41:32.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.329+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4eb8062630 con 0x7f4ea40a3240 2024-09-19T05:41:32.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:32 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T05:41:32.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:32 smithi038 ceph-mon[31227]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:41:32.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.338+0000 7f4eaeffd640 1 -- 172.21.15.38:0/4278733723 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340541 (secure 0 0 0) 0x7f4eb8062630 con 0x7f4ea40a3240 2024-09-19T05:41:32.344 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:32.346 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 msgr2=0x7f4ea40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 0x7f4ea40a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005e00 tx=0x7f4ea8005c50 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 msgr2=0x7f4eb80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c970 tx=0x7f4eb400ce30 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 shutdown_connections 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4ea40a3240 0x7f4ea40a5700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8106f20 0x7f4eb8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81027a0 0x7f4eb8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 --2- 172.21.15.38:0/4278733723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4eb8101df0 0x7f4eb80744f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 >> 172.21.15.38:0/4278733723 conn(0x7f4eb80fd8f0 msgr2=0x7f4eb80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:32.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 shutdown_connections 2024-09-19T05:41:32.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:32.342+0000 7f4ebfed0640 1 -- 172.21.15.38:0/4278733723 wait complete. 2024-09-19T05:41:32.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:32 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T05:41:32.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:32 smithi179 ceph-mon[35781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:41:32.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:32 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2644088895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T05:41:32.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:32 smithi047 ceph-mon[35327]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:41:33.100 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-19T05:41:31.623091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664339,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1195,"num_read_kb":731,"num_write":453,"num_write_kb":3102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1182552,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355424,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586400,"statfs":{"total":1151940427776,"available":1151576473600,"internally_reserved":0,"allocated":14778368,"data_stored":8180863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":76,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:30.253339+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:41:30.253339+0000","last_peered":"2024-09-19T05:41:30.253339+0000","last_clean":"2024-09-19T05:41:30.253339+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:41:30.253339+0000","last_undegraded":"2024-09-19T05:41:30.253339+0000","last_fullsized":"2024-09-19T05:41:30.253339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208532+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:41:11.208532+0000","last_peered":"2024-09-19T05:41:11.208532+0000","last_clean":"2024-09-19T05:41:11.208532+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:41:11.208532+0000","last_undegraded":"2024-09-19T05:41:11.208532+0000","last_fullsized":"2024-09-19T05:41:11.208532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195225+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:41:11.195225+0000","last_peered":"2024-09-19T05:41:11.195225+0000","last_clean":"2024-09-19T05:41:11.195225+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:41:11.195225+0000","last_undegraded":"2024-09-19T05:41:11.195225+0000","last_fullsized":"2024-09-19T05:41:11.195225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209182+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:41:11.209182+0000","last_peered":"2024-09-19T05:41:11.209182+0000","last_clean":"2024-09-19T05:41:11.209182+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:41:11.209182+0000","last_undegraded":"2024-09-19T05:41:11.209182+0000","last_fullsized":"2024-09-19T05:41:11.209182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199152+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:41:11.199152+0000","last_peered":"2024-09-19T05:41:11.199152+0000","last_clean":"2024-09-19T05:41:11.199152+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:41:11.199152+0000","last_undegraded":"2024-09-19T05:41:11.199152+0000","last_fullsized":"2024-09-19T05:41:11.199152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194439+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:41:11.194439+0000","last_peered":"2024-09-19T05:41:11.194439+0000","last_clean":"2024-09-19T05:41:11.194439+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:41:11.194439+0000","last_undegraded":"2024-09-19T05:41:11.194439+0000","last_fullsized":"2024-09-19T05:41:11.194439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.230359+0000","last_change":"2024-09-19T05:41:11.229928+0000","last_active":"2024-09-19T05:41:11.230359+0000","last_peered":"2024-09-19T05:41:11.230359+0000","last_clean":"2024-09-19T05:41:11.230359+0000","last_became_active":"2024-09-19T05:41:11.210845+0000","last_became_peered":"2024-09-19T05:41:11.210845+0000","last_unstale":"2024-09-19T05:41:11.230359+0000","last_undegraded":"2024-09-19T05:41:11.230359+0000","last_fullsized":"2024-09-19T05:41:11.230359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.337244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209586+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:41:11.209586+0000","last_peered":"2024-09-19T05:41:11.209586+0000","last_clean":"2024-09-19T05:41:11.209586+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:41:11.209586+0000","last_undegraded":"2024-09-19T05:41:11.209586+0000","last_fullsized":"2024-09-19T05:41:11.209586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194874+0000","last_change":"2024-09-19T05:40:10.595400+0000","last_active":"2024-09-19T05:41:11.194874+0000","last_peered":"2024-09-19T05:41:11.194874+0000","last_clean":"2024-09-19T05:41:11.194874+0000","last_became_active":"2024-09-19T05:40:10.595015+0000","last_became_peered":"2024-09-19T05:40:10.595015+0000","last_unstale":"2024-09-19T05:41:11.194874+0000","last_undegraded":"2024-09-19T05:41:11.194874+0000","last_fullsized":"2024-09-19T05:41:11.194874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.449136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200251+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:41:11.200251+0000","last_peered":"2024-09-19T05:41:11.200251+0000","last_clean":"2024-09-19T05:41:11.200251+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:41:11.200251+0000","last_undegraded":"2024-09-19T05:41:11.200251+0000","last_fullsized":"2024-09-19T05:41:11.200251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209053+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:41:11.209053+0000","last_peered":"2024-09-19T05:41:11.209053+0000","last_clean":"2024-09-19T05:41:11.209053+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:41:11.209053+0000","last_undegraded":"2024-09-19T05:41:11.209053+0000","last_fullsized":"2024-09-19T05:41:11.209053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194541+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:41:11.194541+0000","last_peered":"2024-09-19T05:41:11.194541+0000","last_clean":"2024-09-19T05:41:11.194541+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:41:11.194541+0000","last_undegraded":"2024-09-19T05:41:11.194541+0000","last_fullsized":"2024-09-19T05:41:11.194541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200008+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:41:11.200008+0000","last_peered":"2024-09-19T05:41:11.200008+0000","last_clean":"2024-09-19T05:41:11.200008+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:41:11.200008+0000","last_undegraded":"2024-09-19T05:41:11.200008+0000","last_fullsized":"2024-09-19T05:41:11.200008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200051+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:41:11.200051+0000","last_peered":"2024-09-19T05:41:11.200051+0000","last_clean":"2024-09-19T05:41:11.200051+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:41:11.200051+0000","last_undegraded":"2024-09-19T05:41:11.200051+0000","last_fullsized":"2024-09-19T05:41:11.200051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194935+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:41:11.194935+0000","last_peered":"2024-09-19T05:41:11.194935+0000","last_clean":"2024-09-19T05:41:11.194935+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:41:11.194935+0000","last_undegraded":"2024-09-19T05:41:11.194935+0000","last_fullsized":"2024-09-19T05:41:11.194935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208569+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:41:11.208569+0000","last_peered":"2024-09-19T05:41:11.208569+0000","last_clean":"2024-09-19T05:41:11.208569+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:41:11.208569+0000","last_undegraded":"2024-09-19T05:41:11.208569+0000","last_fullsized":"2024-09-19T05:41:11.208569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195418+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:41:11.195418+0000","last_peered":"2024-09-19T05:41:11.195418+0000","last_clean":"2024-09-19T05:41:11.195418+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:41:11.195418+0000","last_undegraded":"2024-09-19T05:41:11.195418+0000","last_fullsized":"2024-09-19T05:41:11.195418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195002+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:41:11.195002+0000","last_peered":"2024-09-19T05:41:11.195002+0000","last_clean":"2024-09-19T05:41:11.195002+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:41:11.195002+0000","last_undegraded":"2024-09-19T05:41:11.195002+0000","last_fullsized":"2024-09-19T05:41:11.195002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197505+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:41:11.197505+0000","last_peered":"2024-09-19T05:41:11.197505+0000","last_clean":"2024-09-19T05:41:11.197505+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:41:11.197505+0000","last_undegraded":"2024-09-19T05:41:11.197505+0000","last_fullsized":"2024-09-19T05:41:11.197505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208944+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:41:11.208944+0000","last_peered":"2024-09-19T05:41:11.208944+0000","last_clean":"2024-09-19T05:41:11.208944+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:41:11.208944+0000","last_undegraded":"2024-09-19T05:41:11.208944+0000","last_fullsized":"2024-09-19T05:41:11.208944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199696+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:41:11.199696+0000","last_peered":"2024-09-19T05:41:11.199696+0000","last_clean":"2024-09-19T05:41:11.199696+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:41:11.199696+0000","last_undegraded":"2024-09-19T05:41:11.199696+0000","last_fullsized":"2024-09-19T05:41:11.199696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199690+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:41:11.199690+0000","last_peered":"2024-09-19T05:41:11.199690+0000","last_clean":"2024-09-19T05:41:11.199690+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:41:11.199690+0000","last_undegraded":"2024-09-19T05:41:11.199690+0000","last_fullsized":"2024-09-19T05:41:11.199690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209598+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:41:11.209598+0000","last_peered":"2024-09-19T05:41:11.209598+0000","last_clean":"2024-09-19T05:41:11.209598+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:41:11.209598+0000","last_undegraded":"2024-09-19T05:41:11.209598+0000","last_fullsized":"2024-09-19T05:41:11.209598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195060+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:41:11.195060+0000","last_peered":"2024-09-19T05:41:11.195060+0000","last_clean":"2024-09-19T05:41:11.195060+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:41:11.195060+0000","last_undegraded":"2024-09-19T05:41:11.195060+0000","last_fullsized":"2024-09-19T05:41:11.195060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198875+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:41:11.198875+0000","last_peered":"2024-09-19T05:41:11.198875+0000","last_clean":"2024-09-19T05:41:11.198875+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:41:11.198875+0000","last_undegraded":"2024-09-19T05:41:11.198875+0000","last_fullsized":"2024-09-19T05:41:11.198875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195267+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:41:11.195267+0000","last_peered":"2024-09-19T05:41:11.195267+0000","last_clean":"2024-09-19T05:41:11.195267+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:41:11.195267+0000","last_undegraded":"2024-09-19T05:41:11.195267+0000","last_fullsized":"2024-09-19T05:41:11.195267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195400+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:41:11.195400+0000","last_peered":"2024-09-19T05:41:11.195400+0000","last_clean":"2024-09-19T05:41:11.195400+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:41:11.195400+0000","last_undegraded":"2024-09-19T05:41:11.195400+0000","last_fullsized":"2024-09-19T05:41:11.195400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208869+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:41:11.208869+0000","last_peered":"2024-09-19T05:41:11.208869+0000","last_clean":"2024-09-19T05:41:11.208869+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:41:11.208869+0000","last_undegraded":"2024-09-19T05:41:11.208869+0000","last_fullsized":"2024-09-19T05:41:11.208869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.454620+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:41:11.454620+0000","last_peered":"2024-09-19T05:41:11.454620+0000","last_clean":"2024-09-19T05:41:11.454620+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:41:11.454620+0000","last_undegraded":"2024-09-19T05:41:11.454620+0000","last_fullsized":"2024-09-19T05:41:11.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208319+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:41:11.208319+0000","last_peered":"2024-09-19T05:41:11.208319+0000","last_clean":"2024-09-19T05:41:11.208319+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:41:11.208319+0000","last_undegraded":"2024-09-19T05:41:11.208319+0000","last_fullsized":"2024-09-19T05:41:11.208319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.211599+0000","last_change":"2024-09-19T05:41:11.210967+0000","last_active":"2024-09-19T05:41:11.211599+0000","last_peered":"2024-09-19T05:41:11.211599+0000","last_clean":"2024-09-19T05:41:11.211599+0000","last_became_active":"2024-09-19T05:41:11.201065+0000","last_became_peered":"2024-09-19T05:41:11.201065+0000","last_unstale":"2024-09-19T05:41:11.211599+0000","last_undegraded":"2024-09-19T05:41:11.211599+0000","last_fullsized":"2024-09-19T05:41:11.211599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:54.393420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197544+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:41:11.197544+0000","last_peered":"2024-09-19T05:41:11.197544+0000","last_clean":"2024-09-19T05:41:11.197544+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:41:11.197544+0000","last_undegraded":"2024-09-19T05:41:11.197544+0000","last_fullsized":"2024-09-19T05:41:11.197544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209065+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:41:11.209065+0000","last_peered":"2024-09-19T05:41:11.209065+0000","last_clean":"2024-09-19T05:41:11.209065+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:41:11.209065+0000","last_undegraded":"2024-09-19T05:41:11.209065+0000","last_fullsized":"2024-09-19T05:41:11.209065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199469+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:41:11.199469+0000","last_peered":"2024-09-19T05:41:11.199469+0000","last_clean":"2024-09-19T05:41:11.199469+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:41:11.199469+0000","last_undegraded":"2024-09-19T05:41:11.199469+0000","last_fullsized":"2024-09-19T05:41:11.199469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199830+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:41:11.199830+0000","last_peered":"2024-09-19T05:41:11.199830+0000","last_clean":"2024-09-19T05:41:11.199830+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:41:11.199830+0000","last_undegraded":"2024-09-19T05:41:11.199830+0000","last_fullsized":"2024-09-19T05:41:11.199830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209286+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:41:11.209286+0000","last_peered":"2024-09-19T05:41:11.209286+0000","last_clean":"2024-09-19T05:41:11.209286+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:41:11.209286+0000","last_undegraded":"2024-09-19T05:41:11.209286+0000","last_fullsized":"2024-09-19T05:41:11.209286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:28.251132+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:41:28.251132+0000","last_peered":"2024-09-19T05:41:28.251132+0000","last_clean":"2024-09-19T05:41:28.251132+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:41:28.251132+0000","last_undegraded":"2024-09-19T05:41:28.251132+0000","last_fullsized":"2024-09-19T05:41:28.251132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209476+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:41:11.209476+0000","last_peered":"2024-09-19T05:41:11.209476+0000","last_clean":"2024-09-19T05:41:11.209476+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:41:11.209476+0000","last_undegraded":"2024-09-19T05:41:11.209476+0000","last_fullsized":"2024-09-19T05:41:11.209476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208429+0000","last_change":"2024-09-19T05:40:10.594007+0000","last_active":"2024-09-19T05:41:11.208429+0000","last_peered":"2024-09-19T05:41:11.208429+0000","last_clean":"2024-09-19T05:41:11.208429+0000","last_became_active":"2024-09-19T05:40:10.593493+0000","last_became_peered":"2024-09-19T05:40:10.593493+0000","last_unstale":"2024-09-19T05:41:11.208429+0000","last_undegraded":"2024-09-19T05:41:11.208429+0000","last_fullsized":"2024-09-19T05:41:11.208429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:58.225840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:28.250566+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:41:28.250566+0000","last_peered":"2024-09-19T05:41:28.250566+0000","last_clean":"2024-09-19T05:41:28.250566+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:41:28.250566+0000","last_undegraded":"2024-09-19T05:41:28.250566+0000","last_fullsized":"2024-09-19T05:41:28.250566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209540+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:41:11.209540+0000","last_peered":"2024-09-19T05:41:11.209540+0000","last_clean":"2024-09-19T05:41:11.209540+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:41:11.209540+0000","last_undegraded":"2024-09-19T05:41:11.209540+0000","last_fullsized":"2024-09-19T05:41:11.209540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208853+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:41:11.208853+0000","last_peered":"2024-09-19T05:41:11.208853+0000","last_clean":"2024-09-19T05:41:11.208853+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:41:11.208853+0000","last_undegraded":"2024-09-19T05:41:11.208853+0000","last_fullsized":"2024-09-19T05:41:11.208853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199656+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:41:11.199656+0000","last_peered":"2024-09-19T05:41:11.199656+0000","last_clean":"2024-09-19T05:41:11.199656+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:41:11.199656+0000","last_undegraded":"2024-09-19T05:41:11.199656+0000","last_fullsized":"2024-09-19T05:41:11.199656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209029+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:41:11.209029+0000","last_peered":"2024-09-19T05:41:11.209029+0000","last_clean":"2024-09-19T05:41:11.209029+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:41:11.209029+0000","last_undegraded":"2024-09-19T05:41:11.209029+0000","last_fullsized":"2024-09-19T05:41:11.209029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208986+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:41:11.208986+0000","last_peered":"2024-09-19T05:41:11.208986+0000","last_clean":"2024-09-19T05:41:11.208986+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:41:11.208986+0000","last_undegraded":"2024-09-19T05:41:11.208986+0000","last_fullsized":"2024-09-19T05:41:11.208986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195015+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:41:11.195015+0000","last_peered":"2024-09-19T05:41:11.195015+0000","last_clean":"2024-09-19T05:41:11.195015+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:41:11.195015+0000","last_undegraded":"2024-09-19T05:41:11.195015+0000","last_fullsized":"2024-09-19T05:41:11.195015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199828+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:41:11.199828+0000","last_peered":"2024-09-19T05:41:11.199828+0000","last_clean":"2024-09-19T05:41:11.199828+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:41:11.199828+0000","last_undegraded":"2024-09-19T05:41:11.199828+0000","last_fullsized":"2024-09-19T05:41:11.199828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200179+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:41:11.200179+0000","last_peered":"2024-09-19T05:41:11.200179+0000","last_clean":"2024-09-19T05:41:11.200179+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:41:11.200179+0000","last_undegraded":"2024-09-19T05:41:11.200179+0000","last_fullsized":"2024-09-19T05:41:11.200179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200398+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:41:11.200398+0000","last_peered":"2024-09-19T05:41:11.200398+0000","last_clean":"2024-09-19T05:41:11.200398+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:41:11.200398+0000","last_undegraded":"2024-09-19T05:41:11.200398+0000","last_fullsized":"2024-09-19T05:41:11.200398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:27.229124+0000","last_change":"2024-09-19T05:41:11.205122+0000","last_active":"2024-09-19T05:41:27.229124+0000","last_peered":"2024-09-19T05:41:27.229124+0000","last_clean":"2024-09-19T05:41:27.229124+0000","last_became_active":"2024-09-19T05:41:11.200544+0000","last_became_peered":"2024-09-19T05:41:11.200544+0000","last_unstale":"2024-09-19T05:41:27.229124+0000","last_undegraded":"2024-09-19T05:41:27.229124+0000","last_fullsized":"2024-09-19T05:41:27.229124+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:11.609857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209616+0000","last_change":"2024-09-19T05:40:10.596589+0000","last_active":"2024-09-19T05:41:11.209616+0000","last_peered":"2024-09-19T05:41:11.209616+0000","last_clean":"2024-09-19T05:41:11.209616+0000","last_became_active":"2024-09-19T05:40:10.596087+0000","last_became_peered":"2024-09-19T05:40:10.596087+0000","last_unstale":"2024-09-19T05:41:11.209616+0000","last_undegraded":"2024-09-19T05:41:11.209616+0000","last_fullsized":"2024-09-19T05:41:11.209616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200123+0000","last_change":"2024-09-19T05:40:10.594372+0000","last_active":"2024-09-19T05:41:11.200123+0000","last_peered":"2024-09-19T05:41:11.200123+0000","last_clean":"2024-09-19T05:41:11.200123+0000","last_became_active":"2024-09-19T05:40:10.594018+0000","last_became_peered":"2024-09-19T05:40:10.594018+0000","last_unstale":"2024-09-19T05:41:11.200123+0000","last_undegraded":"2024-09-19T05:41:11.200123+0000","last_fullsized":"2024-09-19T05:41:11.200123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:23.706247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209269+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:41:11.209269+0000","last_peered":"2024-09-19T05:41:11.209269+0000","last_clean":"2024-09-19T05:41:11.209269+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:41:11.209269+0000","last_undegraded":"2024-09-19T05:41:11.209269+0000","last_fullsized":"2024-09-19T05:41:11.209269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198850+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:41:11.198850+0000","last_peered":"2024-09-19T05:41:11.198850+0000","last_clean":"2024-09-19T05:41:11.198850+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:41:11.198850+0000","last_undegraded":"2024-09-19T05:41:11.198850+0000","last_fullsized":"2024-09-19T05:41:11.198850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200345+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:41:11.200345+0000","last_peered":"2024-09-19T05:41:11.200345+0000","last_clean":"2024-09-19T05:41:11.200345+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:41:11.200345+0000","last_undegraded":"2024-09-19T05:41:11.200345+0000","last_fullsized":"2024-09-19T05:41:11.200345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209419+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:41:11.209419+0000","last_peered":"2024-09-19T05:41:11.209419+0000","last_clean":"2024-09-19T05:41:11.209419+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:41:11.209419+0000","last_undegraded":"2024-09-19T05:41:11.209419+0000","last_fullsized":"2024-09-19T05:41:11.209419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208290+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:41:11.208290+0000","last_peered":"2024-09-19T05:41:11.208290+0000","last_clean":"2024-09-19T05:41:11.208290+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:41:11.208290+0000","last_undegraded":"2024-09-19T05:41:11.208290+0000","last_fullsized":"2024-09-19T05:41:11.208290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197736+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:41:11.197736+0000","last_peered":"2024-09-19T05:41:11.197736+0000","last_clean":"2024-09-19T05:41:11.197736+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:41:11.197736+0000","last_undegraded":"2024-09-19T05:41:11.197736+0000","last_fullsized":"2024-09-19T05:41:11.197736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200434+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:41:11.200434+0000","last_peered":"2024-09-19T05:41:11.200434+0000","last_clean":"2024-09-19T05:41:11.200434+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:41:11.200434+0000","last_undegraded":"2024-09-19T05:41:11.200434+0000","last_fullsized":"2024-09-19T05:41:11.200434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208357+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:41:11.208357+0000","last_peered":"2024-09-19T05:41:11.208357+0000","last_clean":"2024-09-19T05:41:11.208357+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:41:11.208357+0000","last_undegraded":"2024-09-19T05:41:11.208357+0000","last_fullsized":"2024-09-19T05:41:11.208357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208646+0000","last_change":"2024-09-19T05:40:10.596504+0000","last_active":"2024-09-19T05:41:11.208646+0000","last_peered":"2024-09-19T05:41:11.208646+0000","last_clean":"2024-09-19T05:41:11.208646+0000","last_became_active":"2024-09-19T05:40:10.596112+0000","last_became_peered":"2024-09-19T05:40:10.596112+0000","last_unstale":"2024-09-19T05:41:11.208646+0000","last_undegraded":"2024-09-19T05:41:11.208646+0000","last_fullsized":"2024-09-19T05:41:11.208646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:02.818567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199743+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:41:11.199743+0000","last_peered":"2024-09-19T05:41:11.199743+0000","last_clean":"2024-09-19T05:41:11.199743+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:41:11.199743+0000","last_undegraded":"2024-09-19T05:41:11.199743+0000","last_fullsized":"2024-09-19T05:41:11.199743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209243+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:41:11.209243+0000","last_peered":"2024-09-19T05:41:11.209243+0000","last_clean":"2024-09-19T05:41:11.209243+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:41:11.209243+0000","last_undegraded":"2024-09-19T05:41:11.209243+0000","last_fullsized":"2024-09-19T05:41:11.209243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199739+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:41:11.199739+0000","last_peered":"2024-09-19T05:41:11.199739+0000","last_clean":"2024-09-19T05:41:11.199739+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:41:11.199739+0000","last_undegraded":"2024-09-19T05:41:11.199739+0000","last_fullsized":"2024-09-19T05:41:11.199739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:27.249628+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:41:27.249628+0000","last_peered":"2024-09-19T05:41:27.249628+0000","last_clean":"2024-09-19T05:41:27.249628+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:41:27.249628+0000","last_undegraded":"2024-09-19T05:41:27.249628+0000","last_fullsized":"2024-09-19T05:41:27.249628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199669+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:41:11.199669+0000","last_peered":"2024-09-19T05:41:11.199669+0000","last_clean":"2024-09-19T05:41:11.199669+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:41:11.199669+0000","last_undegraded":"2024-09-19T05:41:11.199669+0000","last_fullsized":"2024-09-19T05:41:11.199669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199990+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:41:11.199990+0000","last_peered":"2024-09-19T05:41:11.199990+0000","last_clean":"2024-09-19T05:41:11.199990+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:41:11.199990+0000","last_undegraded":"2024-09-19T05:41:11.199990+0000","last_fullsized":"2024-09-19T05:41:11.199990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208852+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:41:11.208852+0000","last_peered":"2024-09-19T05:41:11.208852+0000","last_clean":"2024-09-19T05:41:11.208852+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:41:11.208852+0000","last_undegraded":"2024-09-19T05:41:11.208852+0000","last_fullsized":"2024-09-19T05:41:11.208852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197401+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:41:11.197401+0000","last_peered":"2024-09-19T05:41:11.197401+0000","last_clean":"2024-09-19T05:41:11.197401+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:41:11.197401+0000","last_undegraded":"2024-09-19T05:41:11.197401+0000","last_fullsized":"2024-09-19T05:41:11.197401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209172+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:41:11.209172+0000","last_peered":"2024-09-19T05:41:11.209172+0000","last_clean":"2024-09-19T05:41:11.209172+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:41:11.209172+0000","last_undegraded":"2024-09-19T05:41:11.209172+0000","last_fullsized":"2024-09-19T05:41:11.209172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195638+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:41:11.195638+0000","last_peered":"2024-09-19T05:41:11.195638+0000","last_clean":"2024-09-19T05:41:11.195638+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:41:11.195638+0000","last_undegraded":"2024-09-19T05:41:11.195638+0000","last_fullsized":"2024-09-19T05:41:11.195638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208925+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:41:11.208925+0000","last_peered":"2024-09-19T05:41:11.208925+0000","last_clean":"2024-09-19T05:41:11.208925+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:41:11.208925+0000","last_undegraded":"2024-09-19T05:41:11.208925+0000","last_fullsized":"2024-09-19T05:41:11.208925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200325+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:41:11.200325+0000","last_peered":"2024-09-19T05:41:11.200325+0000","last_clean":"2024-09-19T05:41:11.200325+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:41:11.200325+0000","last_undegraded":"2024-09-19T05:41:11.200325+0000","last_fullsized":"2024-09-19T05:41:11.200325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.259718+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:41:31.259718+0000","last_peered":"2024-09-19T05:41:31.259718+0000","last_clean":"2024-09-19T05:41:31.259718+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:41:31.259718+0000","last_undegraded":"2024-09-19T05:41:31.259718+0000","last_fullsized":"2024-09-19T05:41:31.259718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65179,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198935+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:41:11.198935+0000","last_peered":"2024-09-19T05:41:11.198935+0000","last_clean":"2024-09-19T05:41:11.198935+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:41:11.198935+0000","last_undegraded":"2024-09-19T05:41:11.198935+0000","last_fullsized":"2024-09-19T05:41:11.198935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210003+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:41:11.210003+0000","last_peered":"2024-09-19T05:41:11.210003+0000","last_clean":"2024-09-19T05:41:11.210003+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:41:11.210003+0000","last_undegraded":"2024-09-19T05:41:11.210003+0000","last_fullsized":"2024-09-19T05:41:11.210003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209213+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:41:11.209213+0000","last_peered":"2024-09-19T05:41:11.209213+0000","last_clean":"2024-09-19T05:41:11.209213+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:41:11.209213+0000","last_undegraded":"2024-09-19T05:41:11.209213+0000","last_fullsized":"2024-09-19T05:41:11.209213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194832+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:41:11.194832+0000","last_peered":"2024-09-19T05:41:11.194832+0000","last_clean":"2024-09-19T05:41:11.194832+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:41:11.194832+0000","last_undegraded":"2024-09-19T05:41:11.194832+0000","last_fullsized":"2024-09-19T05:41:11.194832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199324+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:41:11.199324+0000","last_peered":"2024-09-19T05:41:11.199324+0000","last_clean":"2024-09-19T05:41:11.199324+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:41:11.199324+0000","last_undegraded":"2024-09-19T05:41:11.199324+0000","last_fullsized":"2024-09-19T05:41:11.199324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200329+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:41:11.200329+0000","last_peered":"2024-09-19T05:41:11.200329+0000","last_clean":"2024-09-19T05:41:11.200329+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:41:11.200329+0000","last_undegraded":"2024-09-19T05:41:11.200329+0000","last_fullsized":"2024-09-19T05:41:11.200329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208882+0000","last_change":"2024-09-19T05:40:10.893405+0000","last_active":"2024-09-19T05:41:11.208882+0000","last_peered":"2024-09-19T05:41:11.208882+0000","last_clean":"2024-09-19T05:41:11.208882+0000","last_became_active":"2024-09-19T05:40:10.892932+0000","last_became_peered":"2024-09-19T05:40:10.892932+0000","last_unstale":"2024-09-19T05:41:11.208882+0000","last_undegraded":"2024-09-19T05:41:11.208882+0000","last_fullsized":"2024-09-19T05:41:11.208882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:56.153847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210119+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:41:11.210119+0000","last_peered":"2024-09-19T05:41:11.210119+0000","last_clean":"2024-09-19T05:41:11.210119+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:41:11.210119+0000","last_undegraded":"2024-09-19T05:41:11.210119+0000","last_fullsized":"2024-09-19T05:41:11.210119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.235843+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:41:31.235843+0000","last_peered":"2024-09-19T05:41:31.235843+0000","last_clean":"2024-09-19T05:41:31.235843+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:41:31.235843+0000","last_undegraded":"2024-09-19T05:41:31.235843+0000","last_fullsized":"2024-09-19T05:41:31.235843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208451+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:41:11.208451+0000","last_peered":"2024-09-19T05:41:11.208451+0000","last_clean":"2024-09-19T05:41:11.208451+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:41:11.208451+0000","last_undegraded":"2024-09-19T05:41:11.208451+0000","last_fullsized":"2024-09-19T05:41:11.208451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198681+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:41:11.198681+0000","last_peered":"2024-09-19T05:41:11.198681+0000","last_clean":"2024-09-19T05:41:11.198681+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:41:11.198681+0000","last_undegraded":"2024-09-19T05:41:11.198681+0000","last_fullsized":"2024-09-19T05:41:11.198681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198437+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:41:11.198437+0000","last_peered":"2024-09-19T05:41:11.198437+0000","last_clean":"2024-09-19T05:41:11.198437+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:41:11.198437+0000","last_undegraded":"2024-09-19T05:41:11.198437+0000","last_fullsized":"2024-09-19T05:41:11.198437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208655+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:41:11.208655+0000","last_peered":"2024-09-19T05:41:11.208655+0000","last_clean":"2024-09-19T05:41:11.208655+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:41:11.208655+0000","last_undegraded":"2024-09-19T05:41:11.208655+0000","last_fullsized":"2024-09-19T05:41:11.208655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197628+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:41:11.197628+0000","last_peered":"2024-09-19T05:41:11.197628+0000","last_clean":"2024-09-19T05:41:11.197628+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:41:11.197628+0000","last_undegraded":"2024-09-19T05:41:11.197628+0000","last_fullsized":"2024-09-19T05:41:11.197628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208477+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:41:11.208477+0000","last_peered":"2024-09-19T05:41:11.208477+0000","last_clean":"2024-09-19T05:41:11.208477+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:41:11.208477+0000","last_undegraded":"2024-09-19T05:41:11.208477+0000","last_fullsized":"2024-09-19T05:41:11.208477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208971+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:41:11.208971+0000","last_peered":"2024-09-19T05:41:11.208971+0000","last_clean":"2024-09-19T05:41:11.208971+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:41:11.208971+0000","last_undegraded":"2024-09-19T05:41:11.208971+0000","last_fullsized":"2024-09-19T05:41:11.208971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209693+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:41:11.209693+0000","last_peered":"2024-09-19T05:41:11.209693+0000","last_clean":"2024-09-19T05:41:11.209693+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:41:11.209693+0000","last_undegraded":"2024-09-19T05:41:11.209693+0000","last_fullsized":"2024-09-19T05:41:11.209693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208754+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:41:11.208754+0000","last_peered":"2024-09-19T05:41:11.208754+0000","last_clean":"2024-09-19T05:41:11.208754+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:41:11.208754+0000","last_undegraded":"2024-09-19T05:41:11.208754+0000","last_fullsized":"2024-09-19T05:41:11.208754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195629+0000","last_change":"2024-09-19T05:40:10.596620+0000","last_active":"2024-09-19T05:41:11.195629+0000","last_peered":"2024-09-19T05:41:11.195629+0000","last_clean":"2024-09-19T05:41:11.195629+0000","last_became_active":"2024-09-19T05:40:10.596126+0000","last_became_peered":"2024-09-19T05:40:10.596126+0000","last_unstale":"2024-09-19T05:41:11.195629+0000","last_undegraded":"2024-09-19T05:41:11.195629+0000","last_fullsized":"2024-09-19T05:41:11.195629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:16.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":149,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:26.252576+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:41:26.252576+0000","last_peered":"2024-09-19T05:41:26.252576+0000","last_clean":"2024-09-19T05:41:26.252576+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:41:26.252576+0000","last_undegraded":"2024-09-19T05:41:26.252576+0000","last_fullsized":"2024-09-19T05:41:26.252576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199794+0000","last_change":"2024-09-19T05:40:10.590527+0000","last_active":"2024-09-19T05:41:11.199794+0000","last_peered":"2024-09-19T05:41:11.199794+0000","last_clean":"2024-09-19T05:41:11.199794+0000","last_became_active":"2024-09-19T05:40:10.590181+0000","last_became_peered":"2024-09-19T05:40:10.590181+0000","last_unstale":"2024-09-19T05:41:11.199794+0000","last_undegraded":"2024-09-19T05:41:11.199794+0000","last_fullsized":"2024-09-19T05:41:11.199794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:22:55.157416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195758+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:41:11.195758+0000","last_peered":"2024-09-19T05:41:11.195758+0000","last_clean":"2024-09-19T05:41:11.195758+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:41:11.195758+0000","last_undegraded":"2024-09-19T05:41:11.195758+0000","last_fullsized":"2024-09-19T05:41:11.195758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197193+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:41:11.197193+0000","last_peered":"2024-09-19T05:41:11.197193+0000","last_clean":"2024-09-19T05:41:11.197193+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:41:11.197193+0000","last_undegraded":"2024-09-19T05:41:11.197193+0000","last_fullsized":"2024-09-19T05:41:11.197193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195772+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:41:11.195772+0000","last_peered":"2024-09-19T05:41:11.195772+0000","last_clean":"2024-09-19T05:41:11.195772+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:41:11.195772+0000","last_undegraded":"2024-09-19T05:41:11.195772+0000","last_fullsized":"2024-09-19T05:41:11.195772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195298+0000","last_change":"2024-09-19T05:40:10.893573+0000","last_active":"2024-09-19T05:41:11.195298+0000","last_peered":"2024-09-19T05:41:11.195298+0000","last_clean":"2024-09-19T05:41:11.195298+0000","last_became_active":"2024-09-19T05:40:10.893216+0000","last_became_peered":"2024-09-19T05:40:10.893216+0000","last_unstale":"2024-09-19T05:41:11.195298+0000","last_undegraded":"2024-09-19T05:41:11.195298+0000","last_fullsized":"2024-09-19T05:41:11.195298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:50.147170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198667+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:41:11.198667+0000","last_peered":"2024-09-19T05:41:11.198667+0000","last_clean":"2024-09-19T05:41:11.198667+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:41:11.198667+0000","last_undegraded":"2024-09-19T05:41:11.198667+0000","last_fullsized":"2024-09-19T05:41:11.198667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209643+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:41:11.209643+0000","last_peered":"2024-09-19T05:41:11.209643+0000","last_clean":"2024-09-19T05:41:11.209643+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:41:11.209643+0000","last_undegraded":"2024-09-19T05:41:11.209643+0000","last_fullsized":"2024-09-19T05:41:11.209643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198638+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:41:11.198638+0000","last_peered":"2024-09-19T05:41:11.198638+0000","last_clean":"2024-09-19T05:41:11.198638+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:41:11.198638+0000","last_undegraded":"2024-09-19T05:41:11.198638+0000","last_fullsized":"2024-09-19T05:41:11.198638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195938+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:41:11.195938+0000","last_peered":"2024-09-19T05:41:11.195938+0000","last_clean":"2024-09-19T05:41:11.195938+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:41:11.195938+0000","last_undegraded":"2024-09-19T05:41:11.195938+0000","last_fullsized":"2024-09-19T05:41:11.195938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208765+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:41:11.208765+0000","last_peered":"2024-09-19T05:41:11.208765+0000","last_clean":"2024-09-19T05:41:11.208765+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:41:11.208765+0000","last_undegraded":"2024-09-19T05:41:11.208765+0000","last_fullsized":"2024-09-19T05:41:11.208765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208449+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:41:11.208449+0000","last_peered":"2024-09-19T05:41:11.208449+0000","last_clean":"2024-09-19T05:41:11.208449+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:41:11.208449+0000","last_undegraded":"2024-09-19T05:41:11.208449+0000","last_fullsized":"2024-09-19T05:41:11.208449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208803+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:41:11.208803+0000","last_peered":"2024-09-19T05:41:11.208803+0000","last_clean":"2024-09-19T05:41:11.208803+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:41:11.208803+0000","last_undegraded":"2024-09-19T05:41:11.208803+0000","last_fullsized":"2024-09-19T05:41:11.208803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210072+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:41:11.210072+0000","last_peered":"2024-09-19T05:41:11.210072+0000","last_clean":"2024-09-19T05:41:11.210072+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:41:11.210072+0000","last_undegraded":"2024-09-19T05:41:11.210072+0000","last_fullsized":"2024-09-19T05:41:11.210072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209689+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:41:11.209689+0000","last_peered":"2024-09-19T05:41:11.209689+0000","last_clean":"2024-09-19T05:41:11.209689+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:41:11.209689+0000","last_undegraded":"2024-09-19T05:41:11.209689+0000","last_fullsized":"2024-09-19T05:41:11.209689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197591+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:41:11.197591+0000","last_peered":"2024-09-19T05:41:11.197591+0000","last_clean":"2024-09-19T05:41:11.197591+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:41:11.197591+0000","last_undegraded":"2024-09-19T05:41:11.197591+0000","last_fullsized":"2024-09-19T05:41:11.197591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.196030+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:41:11.196030+0000","last_peered":"2024-09-19T05:41:11.196030+0000","last_clean":"2024-09-19T05:41:11.196030+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:41:11.196030+0000","last_undegraded":"2024-09-19T05:41:11.196030+0000","last_fullsized":"2024-09-19T05:41:11.196030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199711+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:41:11.199711+0000","last_peered":"2024-09-19T05:41:11.199711+0000","last_clean":"2024-09-19T05:41:11.199711+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:41:11.199711+0000","last_undegraded":"2024-09-19T05:41:11.199711+0000","last_fullsized":"2024-09-19T05:41:11.199711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197870+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:41:11.197870+0000","last_peered":"2024-09-19T05:41:11.197870+0000","last_clean":"2024-09-19T05:41:11.197870+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:41:11.197870+0000","last_undegraded":"2024-09-19T05:41:11.197870+0000","last_fullsized":"2024-09-19T05:41:11.197870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209348+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:41:11.209348+0000","last_peered":"2024-09-19T05:41:11.209348+0000","last_clean":"2024-09-19T05:41:11.209348+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:41:11.209348+0000","last_undegraded":"2024-09-19T05:41:11.209348+0000","last_fullsized":"2024-09-19T05:41:11.209348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198183+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:41:11.198183+0000","last_peered":"2024-09-19T05:41:11.198183+0000","last_clean":"2024-09-19T05:41:11.198183+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:41:11.198183+0000","last_undegraded":"2024-09-19T05:41:11.198183+0000","last_fullsized":"2024-09-19T05:41:11.198183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198273+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:41:11.198273+0000","last_peered":"2024-09-19T05:41:11.198273+0000","last_clean":"2024-09-19T05:41:11.198273+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:41:11.198273+0000","last_undegraded":"2024-09-19T05:41:11.198273+0000","last_fullsized":"2024-09-19T05:41:11.198273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209399+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:41:11.209399+0000","last_peered":"2024-09-19T05:41:11.209399+0000","last_clean":"2024-09-19T05:41:11.209399+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:41:11.209399+0000","last_undegraded":"2024-09-19T05:41:11.209399+0000","last_fullsized":"2024-09-19T05:41:11.209399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199326+0000","last_change":"2024-09-19T05:40:10.590802+0000","last_active":"2024-09-19T05:41:11.199326+0000","last_peered":"2024-09-19T05:41:11.199326+0000","last_clean":"2024-09-19T05:41:11.199326+0000","last_became_active":"2024-09-19T05:40:10.590515+0000","last_became_peered":"2024-09-19T05:40:10.590515+0000","last_unstale":"2024-09-19T05:41:11.199326+0000","last_undegraded":"2024-09-19T05:41:11.199326+0000","last_fullsized":"2024-09-19T05:41:11.199326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:37.010052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200430+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:41:11.200430+0000","last_peered":"2024-09-19T05:41:11.200430+0000","last_clean":"2024-09-19T05:41:11.200430+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:41:11.200430+0000","last_undegraded":"2024-09-19T05:41:11.200430+0000","last_fullsized":"2024-09-19T05:41:11.200430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209676+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:41:11.209676+0000","last_peered":"2024-09-19T05:41:11.209676+0000","last_clean":"2024-09-19T05:41:11.209676+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:41:11.209676+0000","last_undegraded":"2024-09-19T05:41:11.209676+0000","last_fullsized":"2024-09-19T05:41:11.209676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210113+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:41:11.210113+0000","last_peered":"2024-09-19T05:41:11.210113+0000","last_clean":"2024-09-19T05:41:11.210113+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:41:11.210113+0000","last_undegraded":"2024-09-19T05:41:11.210113+0000","last_fullsized":"2024-09-19T05:41:11.210113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208375+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:41:11.208375+0000","last_peered":"2024-09-19T05:41:11.208375+0000","last_clean":"2024-09-19T05:41:11.208375+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:41:11.208375+0000","last_undegraded":"2024-09-19T05:41:11.208375+0000","last_fullsized":"2024-09-19T05:41:11.208375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199651+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:41:11.199651+0000","last_peered":"2024-09-19T05:41:11.199651+0000","last_clean":"2024-09-19T05:41:11.199651+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:41:11.199651+0000","last_undegraded":"2024-09-19T05:41:11.199651+0000","last_fullsized":"2024-09-19T05:41:11.199651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199857+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:41:11.199857+0000","last_peered":"2024-09-19T05:41:11.199857+0000","last_clean":"2024-09-19T05:41:11.199857+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:41:11.199857+0000","last_undegraded":"2024-09-19T05:41:11.199857+0000","last_fullsized":"2024-09-19T05:41:11.199857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195219+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:41:11.195219+0000","last_peered":"2024-09-19T05:41:11.195219+0000","last_clean":"2024-09-19T05:41:11.195219+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:41:11.195219+0000","last_undegraded":"2024-09-19T05:41:11.195219+0000","last_fullsized":"2024-09-19T05:41:11.195219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198352+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:41:11.198352+0000","last_peered":"2024-09-19T05:41:11.198352+0000","last_clean":"2024-09-19T05:41:11.198352+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:41:11.198352+0000","last_undegraded":"2024-09-19T05:41:11.198352+0000","last_fullsized":"2024-09-19T05:41:11.198352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198813+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:41:11.198813+0000","last_peered":"2024-09-19T05:41:11.198813+0000","last_clean":"2024-09-19T05:41:11.198813+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:41:11.198813+0000","last_undegraded":"2024-09-19T05:41:11.198813+0000","last_fullsized":"2024-09-19T05:41:11.198813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197743+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:41:11.197743+0000","last_peered":"2024-09-19T05:41:11.197743+0000","last_clean":"2024-09-19T05:41:11.197743+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:41:11.197743+0000","last_undegraded":"2024-09-19T05:41:11.197743+0000","last_fullsized":"2024-09-19T05:41:11.197743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73737,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1069,"num_read_kb":622,"num_write":270,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1816,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":518967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1170850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":522347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":518517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1667072,"data_stored":1116337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":514289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1675264,"data_stored":1110851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":520067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":513463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":579240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1118208,"data_stored":580190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":515745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:41:33.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:41:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:33.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:33 smithi038 ceph-mon[31227]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T05:41:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:33 smithi038 ceph-mon[31227]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:33 smithi179 ceph-mon[35781]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T05:41:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:33 smithi179 ceph-mon[35781]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:33 smithi047 ceph-mon[35327]: pgmap v362: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-19T05:41:33.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:33 smithi047 ceph-mon[35327]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:33.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 -- 172.21.15.38:0/2023963153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd18102de0 msgr2=0x7fdd181031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:33.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 --2- 172.21.15.38:0/2023963153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd18102de0 0x7fdd181031c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009900 tx=0x7fdd0802a480 comp rx=0 tx=0).stop 2024-09-19T05:41:33.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 -- 172.21.15.38:0/2023963153 shutdown_connections 2024-09-19T05:41:33.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 --2- 172.21.15.38:0/2023963153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd1810c180 0x7fdd1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:33.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 --2- 172.21.15.38:0/2023963153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 0x7fdd1810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 --2- 172.21.15.38:0/2023963153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd18102de0 0x7fdd181031c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 -- 172.21.15.38:0/2023963153 >> 172.21.15.38:0/2023963153 conn(0x7fdd180fc7a0 msgr2=0x7fdd180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 -- 172.21.15.38:0/2023963153 shutdown_connections 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.870+0000 7fdd1d269640 1 -- 172.21.15.38:0/2023963153 wait complete. 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 Processor -- start 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 -- start start 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd18102de0 0x7fdd181984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 0x7fdd181989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18111fc0 con 0x7fdd1810c180 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18111e40 con 0x7fdd18102de0 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd1d269640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18112140 con 0x7fdd18103700 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.871+0000 7fdd177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59770/0 (socket says 172.21.15.38:59770) 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 -- 172.21.15.38:0/3816909767 learned_addr learned my addr 172.21.15.38:0/3816909767 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd16ffd640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd18102de0 0x7fdd181984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd0ffff640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 0x7fdd181989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 -- 172.21.15.38:0/3816909767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 msgr2=0x7fdd181989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:33.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 0x7fdd181989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 -- 172.21.15.38:0/3816909767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd18102de0 msgr2=0x7fdd181984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd18102de0 0x7fdd181984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 -- 172.21.15.38:0/3816909767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1819d450 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd177fe640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000eaa0 tx=0x7fdd1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000cdb0 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10005020 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100116b0 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1819d740 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.872+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181a5020 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.873+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd181031c0 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.875+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdd10005560 con 0x7fdd1810c180 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.875+0000 7fdd14ff9640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 0x7fdd040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:33.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.875+0000 7fdd16ffd640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 0x7fdd040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:33.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.875+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fdd100d01f0 con 0x7fdd1810c180 2024-09-19T05:41:33.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.876+0000 7fdd16ffd640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 0x7fdd040a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:33.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:33.878+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd10016030 con 0x7fdd1810c180 2024-09-19T05:41:34.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.055+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd18062630 con 0x7fdd040a3240 2024-09-19T05:41:34.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.064+0000 7fdd14ff9640 1 -- 172.21.15.38:0/3816909767 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340542 (secure 0 0 0) 0x7fdd18062630 con 0x7fdd040a3240 2024-09-19T05:41:34.069 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:34.070 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:41:34.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 msgr2=0x7fdd040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:34.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 0x7fdd040a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000c00 tx=0x7fdd08005b50 comp rx=0 tx=0).stop 2024-09-19T05:41:34.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 msgr2=0x7fdd1819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000eaa0 tx=0x7fdd1000ef60 comp rx=0 tx=0).stop 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 shutdown_connections 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdd040a3240 0x7fdd040a5700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd1810c180 0x7fdd1819cd70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd18103700 0x7fdd181989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 --2- 172.21.15.38:0/3816909767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd18102de0 0x7fdd181984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 >> 172.21.15.38:0/3816909767 conn(0x7fdd180fc7a0 msgr2=0x7fdd180fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 shutdown_connections 2024-09-19T05:41:34.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:34.067+0000 7fdd1d269640 1 -- 172.21.15.38:0/3816909767 wait complete. 2024-09-19T05:41:34.740 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-19T05:41:33.623560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1211,"num_read_kb":732,"num_write":456,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1182552,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":874,"ondisk_log_size":874,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355396,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586428,"statfs":{"total":1151940427776,"available":1151576502272,"internally_reserved":0,"allocated":14778368,"data_stored":8182095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":52,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:30.253339+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:41:30.253339+0000","last_peered":"2024-09-19T05:41:30.253339+0000","last_clean":"2024-09-19T05:41:30.253339+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:41:30.253339+0000","last_undegraded":"2024-09-19T05:41:30.253339+0000","last_fullsized":"2024-09-19T05:41:30.253339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208532+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:41:11.208532+0000","last_peered":"2024-09-19T05:41:11.208532+0000","last_clean":"2024-09-19T05:41:11.208532+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:41:11.208532+0000","last_undegraded":"2024-09-19T05:41:11.208532+0000","last_fullsized":"2024-09-19T05:41:11.208532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195298+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:41:11.195298+0000","last_peered":"2024-09-19T05:41:11.195298+0000","last_clean":"2024-09-19T05:41:11.195298+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:41:11.195298+0000","last_undegraded":"2024-09-19T05:41:11.195298+0000","last_fullsized":"2024-09-19T05:41:11.195298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209182+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:41:11.209182+0000","last_peered":"2024-09-19T05:41:11.209182+0000","last_clean":"2024-09-19T05:41:11.209182+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:41:11.209182+0000","last_undegraded":"2024-09-19T05:41:11.209182+0000","last_fullsized":"2024-09-19T05:41:11.209182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199152+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:41:11.199152+0000","last_peered":"2024-09-19T05:41:11.199152+0000","last_clean":"2024-09-19T05:41:11.199152+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:41:11.199152+0000","last_undegraded":"2024-09-19T05:41:11.199152+0000","last_fullsized":"2024-09-19T05:41:11.199152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194520+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:41:11.194520+0000","last_peered":"2024-09-19T05:41:11.194520+0000","last_clean":"2024-09-19T05:41:11.194520+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:41:11.194520+0000","last_undegraded":"2024-09-19T05:41:11.194520+0000","last_fullsized":"2024-09-19T05:41:11.194520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.230359+0000","last_change":"2024-09-19T05:41:11.229928+0000","last_active":"2024-09-19T05:41:11.230359+0000","last_peered":"2024-09-19T05:41:11.230359+0000","last_clean":"2024-09-19T05:41:11.230359+0000","last_became_active":"2024-09-19T05:41:11.210845+0000","last_became_peered":"2024-09-19T05:41:11.210845+0000","last_unstale":"2024-09-19T05:41:11.230359+0000","last_undegraded":"2024-09-19T05:41:11.230359+0000","last_fullsized":"2024-09-19T05:41:11.230359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.337244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209649+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:41:11.209649+0000","last_peered":"2024-09-19T05:41:11.209649+0000","last_clean":"2024-09-19T05:41:11.209649+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:41:11.209649+0000","last_undegraded":"2024-09-19T05:41:11.209649+0000","last_fullsized":"2024-09-19T05:41:11.209649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194874+0000","last_change":"2024-09-19T05:40:10.595400+0000","last_active":"2024-09-19T05:41:11.194874+0000","last_peered":"2024-09-19T05:41:11.194874+0000","last_clean":"2024-09-19T05:41:11.194874+0000","last_became_active":"2024-09-19T05:40:10.595015+0000","last_became_peered":"2024-09-19T05:40:10.595015+0000","last_unstale":"2024-09-19T05:41:11.194874+0000","last_undegraded":"2024-09-19T05:41:11.194874+0000","last_fullsized":"2024-09-19T05:41:11.194874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.449136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200307+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:41:11.200307+0000","last_peered":"2024-09-19T05:41:11.200307+0000","last_clean":"2024-09-19T05:41:11.200307+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:41:11.200307+0000","last_undegraded":"2024-09-19T05:41:11.200307+0000","last_fullsized":"2024-09-19T05:41:11.200307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209105+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:41:11.209105+0000","last_peered":"2024-09-19T05:41:11.209105+0000","last_clean":"2024-09-19T05:41:11.209105+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:41:11.209105+0000","last_undegraded":"2024-09-19T05:41:11.209105+0000","last_fullsized":"2024-09-19T05:41:11.209105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194541+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:41:11.194541+0000","last_peered":"2024-09-19T05:41:11.194541+0000","last_clean":"2024-09-19T05:41:11.194541+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:41:11.194541+0000","last_undegraded":"2024-09-19T05:41:11.194541+0000","last_fullsized":"2024-09-19T05:41:11.194541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200008+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:41:11.200008+0000","last_peered":"2024-09-19T05:41:11.200008+0000","last_clean":"2024-09-19T05:41:11.200008+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:41:11.200008+0000","last_undegraded":"2024-09-19T05:41:11.200008+0000","last_fullsized":"2024-09-19T05:41:11.200008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200051+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:41:11.200051+0000","last_peered":"2024-09-19T05:41:11.200051+0000","last_clean":"2024-09-19T05:41:11.200051+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:41:11.200051+0000","last_undegraded":"2024-09-19T05:41:11.200051+0000","last_fullsized":"2024-09-19T05:41:11.200051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194935+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:41:11.194935+0000","last_peered":"2024-09-19T05:41:11.194935+0000","last_clean":"2024-09-19T05:41:11.194935+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:41:11.194935+0000","last_undegraded":"2024-09-19T05:41:11.194935+0000","last_fullsized":"2024-09-19T05:41:11.194935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208569+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:41:11.208569+0000","last_peered":"2024-09-19T05:41:11.208569+0000","last_clean":"2024-09-19T05:41:11.208569+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:41:11.208569+0000","last_undegraded":"2024-09-19T05:41:11.208569+0000","last_fullsized":"2024-09-19T05:41:11.208569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:13.310224+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:41:13.310224+0000","last_peered":"2024-09-19T05:41:13.310224+0000","last_clean":"2024-09-19T05:41:13.310224+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:41:13.310224+0000","last_undegraded":"2024-09-19T05:41:13.310224+0000","last_fullsized":"2024-09-19T05:41:13.310224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195002+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:41:11.195002+0000","last_peered":"2024-09-19T05:41:11.195002+0000","last_clean":"2024-09-19T05:41:11.195002+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:41:11.195002+0000","last_undegraded":"2024-09-19T05:41:11.195002+0000","last_fullsized":"2024-09-19T05:41:11.195002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197505+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:41:11.197505+0000","last_peered":"2024-09-19T05:41:11.197505+0000","last_clean":"2024-09-19T05:41:11.197505+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:41:11.197505+0000","last_undegraded":"2024-09-19T05:41:11.197505+0000","last_fullsized":"2024-09-19T05:41:11.197505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208944+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:41:11.208944+0000","last_peered":"2024-09-19T05:41:11.208944+0000","last_clean":"2024-09-19T05:41:11.208944+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:41:11.208944+0000","last_undegraded":"2024-09-19T05:41:11.208944+0000","last_fullsized":"2024-09-19T05:41:11.208944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199696+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:41:11.199696+0000","last_peered":"2024-09-19T05:41:11.199696+0000","last_clean":"2024-09-19T05:41:11.199696+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:41:11.199696+0000","last_undegraded":"2024-09-19T05:41:11.199696+0000","last_fullsized":"2024-09-19T05:41:11.199696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199690+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:41:11.199690+0000","last_peered":"2024-09-19T05:41:11.199690+0000","last_clean":"2024-09-19T05:41:11.199690+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:41:11.199690+0000","last_undegraded":"2024-09-19T05:41:11.199690+0000","last_fullsized":"2024-09-19T05:41:11.199690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209598+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:41:11.209598+0000","last_peered":"2024-09-19T05:41:11.209598+0000","last_clean":"2024-09-19T05:41:11.209598+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:41:11.209598+0000","last_undegraded":"2024-09-19T05:41:11.209598+0000","last_fullsized":"2024-09-19T05:41:11.209598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195060+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:41:11.195060+0000","last_peered":"2024-09-19T05:41:11.195060+0000","last_clean":"2024-09-19T05:41:11.195060+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:41:11.195060+0000","last_undegraded":"2024-09-19T05:41:11.195060+0000","last_fullsized":"2024-09-19T05:41:11.195060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198875+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:41:11.198875+0000","last_peered":"2024-09-19T05:41:11.198875+0000","last_clean":"2024-09-19T05:41:11.198875+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:41:11.198875+0000","last_undegraded":"2024-09-19T05:41:11.198875+0000","last_fullsized":"2024-09-19T05:41:11.198875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195267+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:41:11.195267+0000","last_peered":"2024-09-19T05:41:11.195267+0000","last_clean":"2024-09-19T05:41:11.195267+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:41:11.195267+0000","last_undegraded":"2024-09-19T05:41:11.195267+0000","last_fullsized":"2024-09-19T05:41:11.195267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195400+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:41:11.195400+0000","last_peered":"2024-09-19T05:41:11.195400+0000","last_clean":"2024-09-19T05:41:11.195400+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:41:11.195400+0000","last_undegraded":"2024-09-19T05:41:11.195400+0000","last_fullsized":"2024-09-19T05:41:11.195400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208869+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:41:11.208869+0000","last_peered":"2024-09-19T05:41:11.208869+0000","last_clean":"2024-09-19T05:41:11.208869+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:41:11.208869+0000","last_undegraded":"2024-09-19T05:41:11.208869+0000","last_fullsized":"2024-09-19T05:41:11.208869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.454620+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:41:11.454620+0000","last_peered":"2024-09-19T05:41:11.454620+0000","last_clean":"2024-09-19T05:41:11.454620+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:41:11.454620+0000","last_undegraded":"2024-09-19T05:41:11.454620+0000","last_fullsized":"2024-09-19T05:41:11.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208319+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:41:11.208319+0000","last_peered":"2024-09-19T05:41:11.208319+0000","last_clean":"2024-09-19T05:41:11.208319+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:41:11.208319+0000","last_undegraded":"2024-09-19T05:41:11.208319+0000","last_fullsized":"2024-09-19T05:41:11.208319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.211599+0000","last_change":"2024-09-19T05:41:11.210967+0000","last_active":"2024-09-19T05:41:11.211599+0000","last_peered":"2024-09-19T05:41:11.211599+0000","last_clean":"2024-09-19T05:41:11.211599+0000","last_became_active":"2024-09-19T05:41:11.201065+0000","last_became_peered":"2024-09-19T05:41:11.201065+0000","last_unstale":"2024-09-19T05:41:11.211599+0000","last_undegraded":"2024-09-19T05:41:11.211599+0000","last_fullsized":"2024-09-19T05:41:11.211599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:54.393420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197544+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:41:11.197544+0000","last_peered":"2024-09-19T05:41:11.197544+0000","last_clean":"2024-09-19T05:41:11.197544+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:41:11.197544+0000","last_undegraded":"2024-09-19T05:41:11.197544+0000","last_fullsized":"2024-09-19T05:41:11.197544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209065+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:41:11.209065+0000","last_peered":"2024-09-19T05:41:11.209065+0000","last_clean":"2024-09-19T05:41:11.209065+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:41:11.209065+0000","last_undegraded":"2024-09-19T05:41:11.209065+0000","last_fullsized":"2024-09-19T05:41:11.209065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199469+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:41:11.199469+0000","last_peered":"2024-09-19T05:41:11.199469+0000","last_clean":"2024-09-19T05:41:11.199469+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:41:11.199469+0000","last_undegraded":"2024-09-19T05:41:11.199469+0000","last_fullsized":"2024-09-19T05:41:11.199469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199830+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:41:11.199830+0000","last_peered":"2024-09-19T05:41:11.199830+0000","last_clean":"2024-09-19T05:41:11.199830+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:41:11.199830+0000","last_undegraded":"2024-09-19T05:41:11.199830+0000","last_fullsized":"2024-09-19T05:41:11.199830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209286+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:41:11.209286+0000","last_peered":"2024-09-19T05:41:11.209286+0000","last_clean":"2024-09-19T05:41:11.209286+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:41:11.209286+0000","last_undegraded":"2024-09-19T05:41:11.209286+0000","last_fullsized":"2024-09-19T05:41:11.209286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:28.251132+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:41:28.251132+0000","last_peered":"2024-09-19T05:41:28.251132+0000","last_clean":"2024-09-19T05:41:28.251132+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:41:28.251132+0000","last_undegraded":"2024-09-19T05:41:28.251132+0000","last_fullsized":"2024-09-19T05:41:28.251132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209476+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:41:11.209476+0000","last_peered":"2024-09-19T05:41:11.209476+0000","last_clean":"2024-09-19T05:41:11.209476+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:41:11.209476+0000","last_undegraded":"2024-09-19T05:41:11.209476+0000","last_fullsized":"2024-09-19T05:41:11.209476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208429+0000","last_change":"2024-09-19T05:40:10.594007+0000","last_active":"2024-09-19T05:41:11.208429+0000","last_peered":"2024-09-19T05:41:11.208429+0000","last_clean":"2024-09-19T05:41:11.208429+0000","last_became_active":"2024-09-19T05:40:10.593493+0000","last_became_peered":"2024-09-19T05:40:10.593493+0000","last_unstale":"2024-09-19T05:41:11.208429+0000","last_undegraded":"2024-09-19T05:41:11.208429+0000","last_fullsized":"2024-09-19T05:41:11.208429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:58.225840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:28.250566+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:41:28.250566+0000","last_peered":"2024-09-19T05:41:28.250566+0000","last_clean":"2024-09-19T05:41:28.250566+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:41:28.250566+0000","last_undegraded":"2024-09-19T05:41:28.250566+0000","last_fullsized":"2024-09-19T05:41:28.250566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209540+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:41:11.209540+0000","last_peered":"2024-09-19T05:41:11.209540+0000","last_clean":"2024-09-19T05:41:11.209540+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:41:11.209540+0000","last_undegraded":"2024-09-19T05:41:11.209540+0000","last_fullsized":"2024-09-19T05:41:11.209540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208853+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:41:11.208853+0000","last_peered":"2024-09-19T05:41:11.208853+0000","last_clean":"2024-09-19T05:41:11.208853+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:41:11.208853+0000","last_undegraded":"2024-09-19T05:41:11.208853+0000","last_fullsized":"2024-09-19T05:41:11.208853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199656+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:41:11.199656+0000","last_peered":"2024-09-19T05:41:11.199656+0000","last_clean":"2024-09-19T05:41:11.199656+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:41:11.199656+0000","last_undegraded":"2024-09-19T05:41:11.199656+0000","last_fullsized":"2024-09-19T05:41:11.199656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209029+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:41:11.209029+0000","last_peered":"2024-09-19T05:41:11.209029+0000","last_clean":"2024-09-19T05:41:11.209029+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:41:11.209029+0000","last_undegraded":"2024-09-19T05:41:11.209029+0000","last_fullsized":"2024-09-19T05:41:11.209029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208986+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:41:11.208986+0000","last_peered":"2024-09-19T05:41:11.208986+0000","last_clean":"2024-09-19T05:41:11.208986+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:41:11.208986+0000","last_undegraded":"2024-09-19T05:41:11.208986+0000","last_fullsized":"2024-09-19T05:41:11.208986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195015+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:41:11.195015+0000","last_peered":"2024-09-19T05:41:11.195015+0000","last_clean":"2024-09-19T05:41:11.195015+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:41:11.195015+0000","last_undegraded":"2024-09-19T05:41:11.195015+0000","last_fullsized":"2024-09-19T05:41:11.195015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199828+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:41:11.199828+0000","last_peered":"2024-09-19T05:41:11.199828+0000","last_clean":"2024-09-19T05:41:11.199828+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:41:11.199828+0000","last_undegraded":"2024-09-19T05:41:11.199828+0000","last_fullsized":"2024-09-19T05:41:11.199828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200179+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:41:11.200179+0000","last_peered":"2024-09-19T05:41:11.200179+0000","last_clean":"2024-09-19T05:41:11.200179+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:41:11.200179+0000","last_undegraded":"2024-09-19T05:41:11.200179+0000","last_fullsized":"2024-09-19T05:41:11.200179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200398+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:41:11.200398+0000","last_peered":"2024-09-19T05:41:11.200398+0000","last_clean":"2024-09-19T05:41:11.200398+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:41:11.200398+0000","last_undegraded":"2024-09-19T05:41:11.200398+0000","last_fullsized":"2024-09-19T05:41:11.200398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.237830+0000","last_change":"2024-09-19T05:41:11.205122+0000","last_active":"2024-09-19T05:41:32.237830+0000","last_peered":"2024-09-19T05:41:32.237830+0000","last_clean":"2024-09-19T05:41:32.237830+0000","last_became_active":"2024-09-19T05:41:11.200544+0000","last_became_peered":"2024-09-19T05:41:11.200544+0000","last_unstale":"2024-09-19T05:41:32.237830+0000","last_undegraded":"2024-09-19T05:41:32.237830+0000","last_fullsized":"2024-09-19T05:41:32.237830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:11.609857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2012,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209616+0000","last_change":"2024-09-19T05:40:10.596589+0000","last_active":"2024-09-19T05:41:11.209616+0000","last_peered":"2024-09-19T05:41:11.209616+0000","last_clean":"2024-09-19T05:41:11.209616+0000","last_became_active":"2024-09-19T05:40:10.596087+0000","last_became_peered":"2024-09-19T05:40:10.596087+0000","last_unstale":"2024-09-19T05:41:11.209616+0000","last_undegraded":"2024-09-19T05:41:11.209616+0000","last_fullsized":"2024-09-19T05:41:11.209616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200123+0000","last_change":"2024-09-19T05:40:10.594372+0000","last_active":"2024-09-19T05:41:11.200123+0000","last_peered":"2024-09-19T05:41:11.200123+0000","last_clean":"2024-09-19T05:41:11.200123+0000","last_became_active":"2024-09-19T05:40:10.594018+0000","last_became_peered":"2024-09-19T05:40:10.594018+0000","last_unstale":"2024-09-19T05:41:11.200123+0000","last_undegraded":"2024-09-19T05:41:11.200123+0000","last_fullsized":"2024-09-19T05:41:11.200123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:23.706247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209269+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:41:11.209269+0000","last_peered":"2024-09-19T05:41:11.209269+0000","last_clean":"2024-09-19T05:41:11.209269+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:41:11.209269+0000","last_undegraded":"2024-09-19T05:41:11.209269+0000","last_fullsized":"2024-09-19T05:41:11.209269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198850+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:41:11.198850+0000","last_peered":"2024-09-19T05:41:11.198850+0000","last_clean":"2024-09-19T05:41:11.198850+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:41:11.198850+0000","last_undegraded":"2024-09-19T05:41:11.198850+0000","last_fullsized":"2024-09-19T05:41:11.198850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200345+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:41:11.200345+0000","last_peered":"2024-09-19T05:41:11.200345+0000","last_clean":"2024-09-19T05:41:11.200345+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:41:11.200345+0000","last_undegraded":"2024-09-19T05:41:11.200345+0000","last_fullsized":"2024-09-19T05:41:11.200345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209419+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:41:11.209419+0000","last_peered":"2024-09-19T05:41:11.209419+0000","last_clean":"2024-09-19T05:41:11.209419+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:41:11.209419+0000","last_undegraded":"2024-09-19T05:41:11.209419+0000","last_fullsized":"2024-09-19T05:41:11.209419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208290+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:41:11.208290+0000","last_peered":"2024-09-19T05:41:11.208290+0000","last_clean":"2024-09-19T05:41:11.208290+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:41:11.208290+0000","last_undegraded":"2024-09-19T05:41:11.208290+0000","last_fullsized":"2024-09-19T05:41:11.208290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197736+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:41:11.197736+0000","last_peered":"2024-09-19T05:41:11.197736+0000","last_clean":"2024-09-19T05:41:11.197736+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:41:11.197736+0000","last_undegraded":"2024-09-19T05:41:11.197736+0000","last_fullsized":"2024-09-19T05:41:11.197736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200434+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:41:11.200434+0000","last_peered":"2024-09-19T05:41:11.200434+0000","last_clean":"2024-09-19T05:41:11.200434+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:41:11.200434+0000","last_undegraded":"2024-09-19T05:41:11.200434+0000","last_fullsized":"2024-09-19T05:41:11.200434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208357+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:41:11.208357+0000","last_peered":"2024-09-19T05:41:11.208357+0000","last_clean":"2024-09-19T05:41:11.208357+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:41:11.208357+0000","last_undegraded":"2024-09-19T05:41:11.208357+0000","last_fullsized":"2024-09-19T05:41:11.208357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208646+0000","last_change":"2024-09-19T05:40:10.596504+0000","last_active":"2024-09-19T05:41:11.208646+0000","last_peered":"2024-09-19T05:41:11.208646+0000","last_clean":"2024-09-19T05:41:11.208646+0000","last_became_active":"2024-09-19T05:40:10.596112+0000","last_became_peered":"2024-09-19T05:40:10.596112+0000","last_unstale":"2024-09-19T05:41:11.208646+0000","last_undegraded":"2024-09-19T05:41:11.208646+0000","last_fullsized":"2024-09-19T05:41:11.208646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:02.818567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199743+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:41:11.199743+0000","last_peered":"2024-09-19T05:41:11.199743+0000","last_clean":"2024-09-19T05:41:11.199743+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:41:11.199743+0000","last_undegraded":"2024-09-19T05:41:11.199743+0000","last_fullsized":"2024-09-19T05:41:11.199743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209243+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:41:11.209243+0000","last_peered":"2024-09-19T05:41:11.209243+0000","last_clean":"2024-09-19T05:41:11.209243+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:41:11.209243+0000","last_undegraded":"2024-09-19T05:41:11.209243+0000","last_fullsized":"2024-09-19T05:41:11.209243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199739+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:41:11.199739+0000","last_peered":"2024-09-19T05:41:11.199739+0000","last_clean":"2024-09-19T05:41:11.199739+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:41:11.199739+0000","last_undegraded":"2024-09-19T05:41:11.199739+0000","last_fullsized":"2024-09-19T05:41:11.199739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.255893+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:41:32.255893+0000","last_peered":"2024-09-19T05:41:32.255893+0000","last_clean":"2024-09-19T05:41:32.255893+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:41:32.255893+0000","last_undegraded":"2024-09-19T05:41:32.255893+0000","last_fullsized":"2024-09-19T05:41:32.255893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199669+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:41:11.199669+0000","last_peered":"2024-09-19T05:41:11.199669+0000","last_clean":"2024-09-19T05:41:11.199669+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:41:11.199669+0000","last_undegraded":"2024-09-19T05:41:11.199669+0000","last_fullsized":"2024-09-19T05:41:11.199669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199990+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:41:11.199990+0000","last_peered":"2024-09-19T05:41:11.199990+0000","last_clean":"2024-09-19T05:41:11.199990+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:41:11.199990+0000","last_undegraded":"2024-09-19T05:41:11.199990+0000","last_fullsized":"2024-09-19T05:41:11.199990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208852+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:41:11.208852+0000","last_peered":"2024-09-19T05:41:11.208852+0000","last_clean":"2024-09-19T05:41:11.208852+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:41:11.208852+0000","last_undegraded":"2024-09-19T05:41:11.208852+0000","last_fullsized":"2024-09-19T05:41:11.208852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197463+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:41:11.197463+0000","last_peered":"2024-09-19T05:41:11.197463+0000","last_clean":"2024-09-19T05:41:11.197463+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:41:11.197463+0000","last_undegraded":"2024-09-19T05:41:11.197463+0000","last_fullsized":"2024-09-19T05:41:11.197463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209172+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:41:11.209172+0000","last_peered":"2024-09-19T05:41:11.209172+0000","last_clean":"2024-09-19T05:41:11.209172+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:41:11.209172+0000","last_undegraded":"2024-09-19T05:41:11.209172+0000","last_fullsized":"2024-09-19T05:41:11.209172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195638+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:41:11.195638+0000","last_peered":"2024-09-19T05:41:11.195638+0000","last_clean":"2024-09-19T05:41:11.195638+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:41:11.195638+0000","last_undegraded":"2024-09-19T05:41:11.195638+0000","last_fullsized":"2024-09-19T05:41:11.195638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208925+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:41:11.208925+0000","last_peered":"2024-09-19T05:41:11.208925+0000","last_clean":"2024-09-19T05:41:11.208925+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:41:11.208925+0000","last_undegraded":"2024-09-19T05:41:11.208925+0000","last_fullsized":"2024-09-19T05:41:11.208925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200325+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:41:11.200325+0000","last_peered":"2024-09-19T05:41:11.200325+0000","last_clean":"2024-09-19T05:41:11.200325+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:41:11.200325+0000","last_undegraded":"2024-09-19T05:41:11.200325+0000","last_fullsized":"2024-09-19T05:41:11.200325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.259718+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:41:31.259718+0000","last_peered":"2024-09-19T05:41:31.259718+0000","last_clean":"2024-09-19T05:41:31.259718+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:41:31.259718+0000","last_undegraded":"2024-09-19T05:41:31.259718+0000","last_fullsized":"2024-09-19T05:41:31.259718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65179,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198935+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:41:11.198935+0000","last_peered":"2024-09-19T05:41:11.198935+0000","last_clean":"2024-09-19T05:41:11.198935+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:41:11.198935+0000","last_undegraded":"2024-09-19T05:41:11.198935+0000","last_fullsized":"2024-09-19T05:41:11.198935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210003+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:41:11.210003+0000","last_peered":"2024-09-19T05:41:11.210003+0000","last_clean":"2024-09-19T05:41:11.210003+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:41:11.210003+0000","last_undegraded":"2024-09-19T05:41:11.210003+0000","last_fullsized":"2024-09-19T05:41:11.210003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209213+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:41:11.209213+0000","last_peered":"2024-09-19T05:41:11.209213+0000","last_clean":"2024-09-19T05:41:11.209213+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:41:11.209213+0000","last_undegraded":"2024-09-19T05:41:11.209213+0000","last_fullsized":"2024-09-19T05:41:11.209213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194832+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:41:11.194832+0000","last_peered":"2024-09-19T05:41:11.194832+0000","last_clean":"2024-09-19T05:41:11.194832+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:41:11.194832+0000","last_undegraded":"2024-09-19T05:41:11.194832+0000","last_fullsized":"2024-09-19T05:41:11.194832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199324+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:41:11.199324+0000","last_peered":"2024-09-19T05:41:11.199324+0000","last_clean":"2024-09-19T05:41:11.199324+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:41:11.199324+0000","last_undegraded":"2024-09-19T05:41:11.199324+0000","last_fullsized":"2024-09-19T05:41:11.199324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200329+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:41:11.200329+0000","last_peered":"2024-09-19T05:41:11.200329+0000","last_clean":"2024-09-19T05:41:11.200329+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:41:11.200329+0000","last_undegraded":"2024-09-19T05:41:11.200329+0000","last_fullsized":"2024-09-19T05:41:11.200329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208882+0000","last_change":"2024-09-19T05:40:10.893405+0000","last_active":"2024-09-19T05:41:11.208882+0000","last_peered":"2024-09-19T05:41:11.208882+0000","last_clean":"2024-09-19T05:41:11.208882+0000","last_became_active":"2024-09-19T05:40:10.892932+0000","last_became_peered":"2024-09-19T05:40:10.892932+0000","last_unstale":"2024-09-19T05:41:11.208882+0000","last_undegraded":"2024-09-19T05:41:11.208882+0000","last_fullsized":"2024-09-19T05:41:11.208882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:56.153847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210119+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:41:11.210119+0000","last_peered":"2024-09-19T05:41:11.210119+0000","last_clean":"2024-09-19T05:41:11.210119+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:41:11.210119+0000","last_undegraded":"2024-09-19T05:41:11.210119+0000","last_fullsized":"2024-09-19T05:41:11.210119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.235843+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:41:31.235843+0000","last_peered":"2024-09-19T05:41:31.235843+0000","last_clean":"2024-09-19T05:41:31.235843+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:41:31.235843+0000","last_undegraded":"2024-09-19T05:41:31.235843+0000","last_fullsized":"2024-09-19T05:41:31.235843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208507+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:41:11.208507+0000","last_peered":"2024-09-19T05:41:11.208507+0000","last_clean":"2024-09-19T05:41:11.208507+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:41:11.208507+0000","last_undegraded":"2024-09-19T05:41:11.208507+0000","last_fullsized":"2024-09-19T05:41:11.208507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198681+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:41:11.198681+0000","last_peered":"2024-09-19T05:41:11.198681+0000","last_clean":"2024-09-19T05:41:11.198681+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:41:11.198681+0000","last_undegraded":"2024-09-19T05:41:11.198681+0000","last_fullsized":"2024-09-19T05:41:11.198681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198437+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:41:11.198437+0000","last_peered":"2024-09-19T05:41:11.198437+0000","last_clean":"2024-09-19T05:41:11.198437+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:41:11.198437+0000","last_undegraded":"2024-09-19T05:41:11.198437+0000","last_fullsized":"2024-09-19T05:41:11.198437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208655+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:41:11.208655+0000","last_peered":"2024-09-19T05:41:11.208655+0000","last_clean":"2024-09-19T05:41:11.208655+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:41:11.208655+0000","last_undegraded":"2024-09-19T05:41:11.208655+0000","last_fullsized":"2024-09-19T05:41:11.208655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197628+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:41:11.197628+0000","last_peered":"2024-09-19T05:41:11.197628+0000","last_clean":"2024-09-19T05:41:11.197628+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:41:11.197628+0000","last_undegraded":"2024-09-19T05:41:11.197628+0000","last_fullsized":"2024-09-19T05:41:11.197628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208477+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:41:11.208477+0000","last_peered":"2024-09-19T05:41:11.208477+0000","last_clean":"2024-09-19T05:41:11.208477+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:41:11.208477+0000","last_undegraded":"2024-09-19T05:41:11.208477+0000","last_fullsized":"2024-09-19T05:41:11.208477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208971+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:41:11.208971+0000","last_peered":"2024-09-19T05:41:11.208971+0000","last_clean":"2024-09-19T05:41:11.208971+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:41:11.208971+0000","last_undegraded":"2024-09-19T05:41:11.208971+0000","last_fullsized":"2024-09-19T05:41:11.208971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209693+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:41:11.209693+0000","last_peered":"2024-09-19T05:41:11.209693+0000","last_clean":"2024-09-19T05:41:11.209693+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:41:11.209693+0000","last_undegraded":"2024-09-19T05:41:11.209693+0000","last_fullsized":"2024-09-19T05:41:11.209693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208811+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:41:11.208811+0000","last_peered":"2024-09-19T05:41:11.208811+0000","last_clean":"2024-09-19T05:41:11.208811+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:41:11.208811+0000","last_undegraded":"2024-09-19T05:41:11.208811+0000","last_fullsized":"2024-09-19T05:41:11.208811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195629+0000","last_change":"2024-09-19T05:40:10.596620+0000","last_active":"2024-09-19T05:41:11.195629+0000","last_peered":"2024-09-19T05:41:11.195629+0000","last_clean":"2024-09-19T05:41:11.195629+0000","last_became_active":"2024-09-19T05:40:10.596126+0000","last_became_peered":"2024-09-19T05:40:10.596126+0000","last_unstale":"2024-09-19T05:41:11.195629+0000","last_undegraded":"2024-09-19T05:41:11.195629+0000","last_fullsized":"2024-09-19T05:41:11.195629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:16.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.260880+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:41:32.260880+0000","last_peered":"2024-09-19T05:41:32.260880+0000","last_clean":"2024-09-19T05:41:32.260880+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:41:32.260880+0000","last_undegraded":"2024-09-19T05:41:32.260880+0000","last_fullsized":"2024-09-19T05:41:32.260880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199935+0000","last_change":"2024-09-19T05:40:10.590527+0000","last_active":"2024-09-19T05:41:11.199935+0000","last_peered":"2024-09-19T05:41:11.199935+0000","last_clean":"2024-09-19T05:41:11.199935+0000","last_became_active":"2024-09-19T05:40:10.590181+0000","last_became_peered":"2024-09-19T05:40:10.590181+0000","last_unstale":"2024-09-19T05:41:11.199935+0000","last_undegraded":"2024-09-19T05:41:11.199935+0000","last_fullsized":"2024-09-19T05:41:11.199935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:22:55.157416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195758+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:41:11.195758+0000","last_peered":"2024-09-19T05:41:11.195758+0000","last_clean":"2024-09-19T05:41:11.195758+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:41:11.195758+0000","last_undegraded":"2024-09-19T05:41:11.195758+0000","last_fullsized":"2024-09-19T05:41:11.195758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197193+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:41:11.197193+0000","last_peered":"2024-09-19T05:41:11.197193+0000","last_clean":"2024-09-19T05:41:11.197193+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:41:11.197193+0000","last_undegraded":"2024-09-19T05:41:11.197193+0000","last_fullsized":"2024-09-19T05:41:11.197193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195772+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:41:11.195772+0000","last_peered":"2024-09-19T05:41:11.195772+0000","last_clean":"2024-09-19T05:41:11.195772+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:41:11.195772+0000","last_undegraded":"2024-09-19T05:41:11.195772+0000","last_fullsized":"2024-09-19T05:41:11.195772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195298+0000","last_change":"2024-09-19T05:40:10.893573+0000","last_active":"2024-09-19T05:41:11.195298+0000","last_peered":"2024-09-19T05:41:11.195298+0000","last_clean":"2024-09-19T05:41:11.195298+0000","last_became_active":"2024-09-19T05:40:10.893216+0000","last_became_peered":"2024-09-19T05:40:10.893216+0000","last_unstale":"2024-09-19T05:41:11.195298+0000","last_undegraded":"2024-09-19T05:41:11.195298+0000","last_fullsized":"2024-09-19T05:41:11.195298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:50.147170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198667+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:41:11.198667+0000","last_peered":"2024-09-19T05:41:11.198667+0000","last_clean":"2024-09-19T05:41:11.198667+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:41:11.198667+0000","last_undegraded":"2024-09-19T05:41:11.198667+0000","last_fullsized":"2024-09-19T05:41:11.198667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209643+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:41:11.209643+0000","last_peered":"2024-09-19T05:41:11.209643+0000","last_clean":"2024-09-19T05:41:11.209643+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:41:11.209643+0000","last_undegraded":"2024-09-19T05:41:11.209643+0000","last_fullsized":"2024-09-19T05:41:11.209643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198638+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:41:11.198638+0000","last_peered":"2024-09-19T05:41:11.198638+0000","last_clean":"2024-09-19T05:41:11.198638+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:41:11.198638+0000","last_undegraded":"2024-09-19T05:41:11.198638+0000","last_fullsized":"2024-09-19T05:41:11.198638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195938+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:41:11.195938+0000","last_peered":"2024-09-19T05:41:11.195938+0000","last_clean":"2024-09-19T05:41:11.195938+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:41:11.195938+0000","last_undegraded":"2024-09-19T05:41:11.195938+0000","last_fullsized":"2024-09-19T05:41:11.195938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208765+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:41:11.208765+0000","last_peered":"2024-09-19T05:41:11.208765+0000","last_clean":"2024-09-19T05:41:11.208765+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:41:11.208765+0000","last_undegraded":"2024-09-19T05:41:11.208765+0000","last_fullsized":"2024-09-19T05:41:11.208765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208449+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:41:11.208449+0000","last_peered":"2024-09-19T05:41:11.208449+0000","last_clean":"2024-09-19T05:41:11.208449+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:41:11.208449+0000","last_undegraded":"2024-09-19T05:41:11.208449+0000","last_fullsized":"2024-09-19T05:41:11.208449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208803+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:41:11.208803+0000","last_peered":"2024-09-19T05:41:11.208803+0000","last_clean":"2024-09-19T05:41:11.208803+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:41:11.208803+0000","last_undegraded":"2024-09-19T05:41:11.208803+0000","last_fullsized":"2024-09-19T05:41:11.208803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210072+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:41:11.210072+0000","last_peered":"2024-09-19T05:41:11.210072+0000","last_clean":"2024-09-19T05:41:11.210072+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:41:11.210072+0000","last_undegraded":"2024-09-19T05:41:11.210072+0000","last_fullsized":"2024-09-19T05:41:11.210072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209689+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:41:11.209689+0000","last_peered":"2024-09-19T05:41:11.209689+0000","last_clean":"2024-09-19T05:41:11.209689+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:41:11.209689+0000","last_undegraded":"2024-09-19T05:41:11.209689+0000","last_fullsized":"2024-09-19T05:41:11.209689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197591+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:41:11.197591+0000","last_peered":"2024-09-19T05:41:11.197591+0000","last_clean":"2024-09-19T05:41:11.197591+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:41:11.197591+0000","last_undegraded":"2024-09-19T05:41:11.197591+0000","last_fullsized":"2024-09-19T05:41:11.197591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.196030+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:41:11.196030+0000","last_peered":"2024-09-19T05:41:11.196030+0000","last_clean":"2024-09-19T05:41:11.196030+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:41:11.196030+0000","last_undegraded":"2024-09-19T05:41:11.196030+0000","last_fullsized":"2024-09-19T05:41:11.196030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199711+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:41:11.199711+0000","last_peered":"2024-09-19T05:41:11.199711+0000","last_clean":"2024-09-19T05:41:11.199711+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:41:11.199711+0000","last_undegraded":"2024-09-19T05:41:11.199711+0000","last_fullsized":"2024-09-19T05:41:11.199711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197870+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:41:11.197870+0000","last_peered":"2024-09-19T05:41:11.197870+0000","last_clean":"2024-09-19T05:41:11.197870+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:41:11.197870+0000","last_undegraded":"2024-09-19T05:41:11.197870+0000","last_fullsized":"2024-09-19T05:41:11.197870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209348+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:41:11.209348+0000","last_peered":"2024-09-19T05:41:11.209348+0000","last_clean":"2024-09-19T05:41:11.209348+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:41:11.209348+0000","last_undegraded":"2024-09-19T05:41:11.209348+0000","last_fullsized":"2024-09-19T05:41:11.209348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198183+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:41:11.198183+0000","last_peered":"2024-09-19T05:41:11.198183+0000","last_clean":"2024-09-19T05:41:11.198183+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:41:11.198183+0000","last_undegraded":"2024-09-19T05:41:11.198183+0000","last_fullsized":"2024-09-19T05:41:11.198183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198273+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:41:11.198273+0000","last_peered":"2024-09-19T05:41:11.198273+0000","last_clean":"2024-09-19T05:41:11.198273+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:41:11.198273+0000","last_undegraded":"2024-09-19T05:41:11.198273+0000","last_fullsized":"2024-09-19T05:41:11.198273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209399+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:41:11.209399+0000","last_peered":"2024-09-19T05:41:11.209399+0000","last_clean":"2024-09-19T05:41:11.209399+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:41:11.209399+0000","last_undegraded":"2024-09-19T05:41:11.209399+0000","last_fullsized":"2024-09-19T05:41:11.209399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199326+0000","last_change":"2024-09-19T05:40:10.590802+0000","last_active":"2024-09-19T05:41:11.199326+0000","last_peered":"2024-09-19T05:41:11.199326+0000","last_clean":"2024-09-19T05:41:11.199326+0000","last_became_active":"2024-09-19T05:40:10.590515+0000","last_became_peered":"2024-09-19T05:40:10.590515+0000","last_unstale":"2024-09-19T05:41:11.199326+0000","last_undegraded":"2024-09-19T05:41:11.199326+0000","last_fullsized":"2024-09-19T05:41:11.199326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:37.010052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200430+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:41:11.200430+0000","last_peered":"2024-09-19T05:41:11.200430+0000","last_clean":"2024-09-19T05:41:11.200430+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:41:11.200430+0000","last_undegraded":"2024-09-19T05:41:11.200430+0000","last_fullsized":"2024-09-19T05:41:11.200430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209676+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:41:11.209676+0000","last_peered":"2024-09-19T05:41:11.209676+0000","last_clean":"2024-09-19T05:41:11.209676+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:41:11.209676+0000","last_undegraded":"2024-09-19T05:41:11.209676+0000","last_fullsized":"2024-09-19T05:41:11.209676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210113+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:41:11.210113+0000","last_peered":"2024-09-19T05:41:11.210113+0000","last_clean":"2024-09-19T05:41:11.210113+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:41:11.210113+0000","last_undegraded":"2024-09-19T05:41:11.210113+0000","last_fullsized":"2024-09-19T05:41:11.210113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208426+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:41:11.208426+0000","last_peered":"2024-09-19T05:41:11.208426+0000","last_clean":"2024-09-19T05:41:11.208426+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:41:11.208426+0000","last_undegraded":"2024-09-19T05:41:11.208426+0000","last_fullsized":"2024-09-19T05:41:11.208426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199651+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:41:11.199651+0000","last_peered":"2024-09-19T05:41:11.199651+0000","last_clean":"2024-09-19T05:41:11.199651+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:41:11.199651+0000","last_undegraded":"2024-09-19T05:41:11.199651+0000","last_fullsized":"2024-09-19T05:41:11.199651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199916+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:41:11.199916+0000","last_peered":"2024-09-19T05:41:11.199916+0000","last_clean":"2024-09-19T05:41:11.199916+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:41:11.199916+0000","last_undegraded":"2024-09-19T05:41:11.199916+0000","last_fullsized":"2024-09-19T05:41:11.199916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195219+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:41:11.195219+0000","last_peered":"2024-09-19T05:41:11.195219+0000","last_clean":"2024-09-19T05:41:11.195219+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:41:11.195219+0000","last_undegraded":"2024-09-19T05:41:11.195219+0000","last_fullsized":"2024-09-19T05:41:11.195219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198352+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:41:11.198352+0000","last_peered":"2024-09-19T05:41:11.198352+0000","last_clean":"2024-09-19T05:41:11.198352+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:41:11.198352+0000","last_undegraded":"2024-09-19T05:41:11.198352+0000","last_fullsized":"2024-09-19T05:41:11.198352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198813+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:41:11.198813+0000","last_peered":"2024-09-19T05:41:11.198813+0000","last_clean":"2024-09-19T05:41:11.198813+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:41:11.198813+0000","last_undegraded":"2024-09-19T05:41:11.198813+0000","last_fullsized":"2024-09-19T05:41:11.198813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197743+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:41:11.197743+0000","last_peered":"2024-09-19T05:41:11.197743+0000","last_clean":"2024-09-19T05:41:11.197743+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:41:11.197743+0000","last_undegraded":"2024-09-19T05:41:11.197743+0000","last_fullsized":"2024-09-19T05:41:11.197743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1085,"num_read_kb":623,"num_write":273,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1816,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":230455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547242,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":519275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1170850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":522347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168635,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":518517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1667072,"data_stored":1116337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":514289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1675264,"data_stored":1110851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":520375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":513463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":579548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":580498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":515745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:41:34.746 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:41:34.746 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:41:35.001 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:35 smithi038 ceph-mon[31227]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:41:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:35 smithi038 ceph-mon[31227]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 -- 172.21.15.38:0/913724544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 msgr2=0x7f83f8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 --2- 172.21.15.38:0/913724544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f8103120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009900 tx=0x7f83f402a480 comp rx=0 tx=0).stop 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 -- 172.21.15.38:0/913724544 shutdown_connections 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 --2- 172.21.15.38:0/913724544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 0x7f83f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 --2- 172.21.15.38:0/913724544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 0x7f83f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 --2- 172.21.15.38:0/913724544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f8103120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 -- 172.21.15.38:0/913724544 >> 172.21.15.38:0/913724544 conn(0x7f83f80fc7a0 msgr2=0x7f83f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 -- 172.21.15.38:0/913724544 shutdown_connections 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.529+0000 7f840074d640 1 -- 172.21.15.38:0/913724544 wait complete. 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 Processor -- start 2024-09-19T05:41:35.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 -- start start 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 0x7f83f8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 0x7f83f819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8111f70 con 0x7f83f8102d40 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8111df0 con 0x7f83f810c170 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f840074d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81120f0 con 0x7f83f8103660 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.530+0000 7f83fe4e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59790/0 (socket says 172.21.15.38:59790) 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 -- 172.21.15.38:0/2003597704 learned_addr learned my addr 172.21.15.38:0/2003597704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fecea640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 0x7f83f819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fdce8640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 0x7f83f8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 -- 172.21.15.38:0/2003597704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 msgr2=0x7f83f8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:35.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 0x7f83f8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 -- 172.21.15.38:0/2003597704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 msgr2=0x7f83f819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 0x7f83f819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 -- 172.21.15.38:0/2003597704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819d4a0 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83fe4e9640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f403f030 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f4003ce0 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819d730 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4035070 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.531+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a5070 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.532+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f8103120 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.533+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f83f4042030 con 0x7f83f8102d40 2024-09-19T05:41:35.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.534+0000 7f83eb7fe640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 0x7f83e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:35.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.534+0000 7f83fdce8640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 0x7f83e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:35.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.534+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f83f40f04f0 con 0x7f83f8102d40 2024-09-19T05:41:35.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.535+0000 7f83fdce8640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 0x7f83e40a5780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83f810d640 tx=0x7f83ec00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:35.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.537+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83f40bc8c0 con 0x7f83f8102d40 2024-09-19T05:41:35.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:35 smithi179 ceph-mon[35781]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:41:35.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:35 smithi179 ceph-mon[35781]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:35.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.712+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 --> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83f8062630 con 0x7f83e40a32c0 2024-09-19T05:41:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.720+0000 7f83eb7fe640 1 -- 172.21.15.38:0/2003597704 <== mgr.14150 v2:172.21.15.38:6800/4137587885 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340542 (secure 0 0 0) 0x7f83f8062630 con 0x7f83e40a32c0 2024-09-19T05:41:35.725 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-19T05:41:35.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:35 smithi047 ceph-mon[35327]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:41:35.729 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:35 smithi047 ceph-mon[35327]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 msgr2=0x7f83e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 0x7f83e40a5780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83f810d640 tx=0x7f83ec00b2e0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 msgr2=0x7f83f81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006010 tx=0x7f83f40042f0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 shutdown_connections 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f83e40a32c0 0x7f83e40a5780 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83f810c170 0x7f83f819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f8103660 0x7f83f8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 --2- 172.21.15.38:0/2003597704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83f8102d40 0x7f83f81984f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 >> 172.21.15.38:0/2003597704 conn(0x7f83f80fc7a0 msgr2=0x7f83f80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.723+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 shutdown_connections 2024-09-19T05:41:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:35.724+0000 7f840074d640 1 -- 172.21.15.38:0/2003597704 wait complete. 2024-09-19T05:41:36.478 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-19T05:41:35.624083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664647,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1221,"num_read_kb":737,"num_write":456,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1182552,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":874,"ondisk_log_size":874,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355396,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586428,"statfs":{"total":1151940427776,"available":1151576502272,"internally_reserved":0,"allocated":14778368,"data_stored":8182403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":348894523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003391"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:30.253339+0000","last_change":"2024-09-19T05:38:59.852164+0000","last_active":"2024-09-19T05:41:30.253339+0000","last_peered":"2024-09-19T05:41:30.253339+0000","last_clean":"2024-09-19T05:41:30.253339+0000","last_became_active":"2024-09-19T05:38:59.851871+0000","last_became_peered":"2024-09-19T05:38:59.851871+0000","last_unstale":"2024-09-19T05:41:30.253339+0000","last_undegraded":"2024-09-19T05:41:30.253339+0000","last_fullsized":"2024-09-19T05:41:30.253339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:09.134080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208532+0000","last_change":"2024-09-19T05:39:01.892314+0000","last_active":"2024-09-19T05:41:11.208532+0000","last_peered":"2024-09-19T05:41:11.208532+0000","last_clean":"2024-09-19T05:41:11.208532+0000","last_became_active":"2024-09-19T05:39:01.891959+0000","last_became_peered":"2024-09-19T05:39:01.891959+0000","last_unstale":"2024-09-19T05:41:11.208532+0000","last_undegraded":"2024-09-19T05:41:11.208532+0000","last_fullsized":"2024-09-19T05:41:11.208532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:12.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195298+0000","last_change":"2024-09-19T05:39:08.953757+0000","last_active":"2024-09-19T05:41:11.195298+0000","last_peered":"2024-09-19T05:41:11.195298+0000","last_clean":"2024-09-19T05:41:11.195298+0000","last_became_active":"2024-09-19T05:39:08.953164+0000","last_became_peered":"2024-09-19T05:39:08.953164+0000","last_unstale":"2024-09-19T05:41:11.195298+0000","last_undegraded":"2024-09-19T05:41:11.195298+0000","last_fullsized":"2024-09-19T05:41:11.195298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:08.491104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209182+0000","last_change":"2024-09-19T05:39:00.095687+0000","last_active":"2024-09-19T05:41:11.209182+0000","last_peered":"2024-09-19T05:41:11.209182+0000","last_clean":"2024-09-19T05:41:11.209182+0000","last_became_active":"2024-09-19T05:39:00.095358+0000","last_became_peered":"2024-09-19T05:39:00.095358+0000","last_unstale":"2024-09-19T05:41:11.209182+0000","last_undegraded":"2024-09-19T05:41:11.209182+0000","last_fullsized":"2024-09-19T05:41:11.209182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:31.777536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199152+0000","last_change":"2024-09-19T05:39:01.895435+0000","last_active":"2024-09-19T05:41:11.199152+0000","last_peered":"2024-09-19T05:41:11.199152+0000","last_clean":"2024-09-19T05:41:11.199152+0000","last_became_active":"2024-09-19T05:39:01.895079+0000","last_became_peered":"2024-09-19T05:39:01.895079+0000","last_unstale":"2024-09-19T05:41:11.199152+0000","last_undegraded":"2024-09-19T05:41:11.199152+0000","last_fullsized":"2024-09-19T05:41:11.199152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:30.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194520+0000","last_change":"2024-09-19T05:39:08.949679+0000","last_active":"2024-09-19T05:41:11.194520+0000","last_peered":"2024-09-19T05:41:11.194520+0000","last_clean":"2024-09-19T05:41:11.194520+0000","last_became_active":"2024-09-19T05:39:08.949266+0000","last_became_peered":"2024-09-19T05:39:08.949266+0000","last_unstale":"2024-09-19T05:41:11.194520+0000","last_undegraded":"2024-09-19T05:41:11.194520+0000","last_fullsized":"2024-09-19T05:41:11.194520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:28.322353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.230359+0000","last_change":"2024-09-19T05:41:11.229928+0000","last_active":"2024-09-19T05:41:11.230359+0000","last_peered":"2024-09-19T05:41:11.230359+0000","last_clean":"2024-09-19T05:41:11.230359+0000","last_became_active":"2024-09-19T05:41:11.210845+0000","last_became_peered":"2024-09-19T05:41:11.210845+0000","last_unstale":"2024-09-19T05:41:11.230359+0000","last_undegraded":"2024-09-19T05:41:11.230359+0000","last_fullsized":"2024-09-19T05:41:11.230359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.337244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209649+0000","last_change":"2024-09-19T05:39:01.902739+0000","last_active":"2024-09-19T05:41:11.209649+0000","last_peered":"2024-09-19T05:41:11.209649+0000","last_clean":"2024-09-19T05:41:11.209649+0000","last_became_active":"2024-09-19T05:39:01.902480+0000","last_became_peered":"2024-09-19T05:39:01.902480+0000","last_unstale":"2024-09-19T05:41:11.209649+0000","last_undegraded":"2024-09-19T05:41:11.209649+0000","last_fullsized":"2024-09-19T05:41:11.209649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:16.843231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194874+0000","last_change":"2024-09-19T05:40:10.595400+0000","last_active":"2024-09-19T05:41:11.194874+0000","last_peered":"2024-09-19T05:41:11.194874+0000","last_clean":"2024-09-19T05:41:11.194874+0000","last_became_active":"2024-09-19T05:40:10.595015+0000","last_became_peered":"2024-09-19T05:40:10.595015+0000","last_unstale":"2024-09-19T05:41:11.194874+0000","last_undegraded":"2024-09-19T05:41:11.194874+0000","last_fullsized":"2024-09-19T05:41:11.194874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:51.449136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200307+0000","last_change":"2024-09-19T05:38:59.854805+0000","last_active":"2024-09-19T05:41:11.200307+0000","last_peered":"2024-09-19T05:41:11.200307+0000","last_clean":"2024-09-19T05:41:11.200307+0000","last_became_active":"2024-09-19T05:38:59.854443+0000","last_became_peered":"2024-09-19T05:38:59.854443+0000","last_unstale":"2024-09-19T05:41:11.200307+0000","last_undegraded":"2024-09-19T05:41:11.200307+0000","last_fullsized":"2024-09-19T05:41:11.200307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:15.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209105+0000","last_change":"2024-09-19T05:39:01.898223+0000","last_active":"2024-09-19T05:41:11.209105+0000","last_peered":"2024-09-19T05:41:11.209105+0000","last_clean":"2024-09-19T05:41:11.209105+0000","last_became_active":"2024-09-19T05:39:01.897455+0000","last_became_peered":"2024-09-19T05:39:01.897455+0000","last_unstale":"2024-09-19T05:41:11.209105+0000","last_undegraded":"2024-09-19T05:41:11.209105+0000","last_fullsized":"2024-09-19T05:41:11.209105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:02.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194541+0000","last_change":"2024-09-19T05:39:08.953732+0000","last_active":"2024-09-19T05:41:11.194541+0000","last_peered":"2024-09-19T05:41:11.194541+0000","last_clean":"2024-09-19T05:41:11.194541+0000","last_became_active":"2024-09-19T05:39:08.953304+0000","last_became_peered":"2024-09-19T05:39:08.953304+0000","last_unstale":"2024-09-19T05:41:11.194541+0000","last_undegraded":"2024-09-19T05:41:11.194541+0000","last_fullsized":"2024-09-19T05:41:11.194541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200008+0000","last_change":"2024-09-19T05:39:01.895584+0000","last_active":"2024-09-19T05:41:11.200008+0000","last_peered":"2024-09-19T05:41:11.200008+0000","last_clean":"2024-09-19T05:41:11.200008+0000","last_became_active":"2024-09-19T05:39:01.895208+0000","last_became_peered":"2024-09-19T05:39:01.895208+0000","last_unstale":"2024-09-19T05:41:11.200008+0000","last_undegraded":"2024-09-19T05:41:11.200008+0000","last_fullsized":"2024-09-19T05:41:11.200008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:02.727725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200051+0000","last_change":"2024-09-19T05:38:59.855349+0000","last_active":"2024-09-19T05:41:11.200051+0000","last_peered":"2024-09-19T05:41:11.200051+0000","last_clean":"2024-09-19T05:41:11.200051+0000","last_became_active":"2024-09-19T05:38:59.855005+0000","last_became_peered":"2024-09-19T05:38:59.855005+0000","last_unstale":"2024-09-19T05:41:11.200051+0000","last_undegraded":"2024-09-19T05:41:11.200051+0000","last_fullsized":"2024-09-19T05:41:11.200051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:49.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194935+0000","last_change":"2024-09-19T05:39:09.048713+0000","last_active":"2024-09-19T05:41:11.194935+0000","last_peered":"2024-09-19T05:41:11.194935+0000","last_clean":"2024-09-19T05:41:11.194935+0000","last_became_active":"2024-09-19T05:39:09.048235+0000","last_became_peered":"2024-09-19T05:39:09.048235+0000","last_unstale":"2024-09-19T05:41:11.194935+0000","last_undegraded":"2024-09-19T05:41:11.194935+0000","last_fullsized":"2024-09-19T05:41:11.194935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:09.846917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208569+0000","last_change":"2024-09-19T05:39:01.898992+0000","last_active":"2024-09-19T05:41:11.208569+0000","last_peered":"2024-09-19T05:41:11.208569+0000","last_clean":"2024-09-19T05:41:11.208569+0000","last_became_active":"2024-09-19T05:39:01.897694+0000","last_became_peered":"2024-09-19T05:39:01.897694+0000","last_unstale":"2024-09-19T05:41:11.208569+0000","last_undegraded":"2024-09-19T05:41:11.208569+0000","last_fullsized":"2024-09-19T05:41:11.208569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:45.018296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:13.310224+0000","last_change":"2024-09-19T05:38:59.857592+0000","last_active":"2024-09-19T05:41:13.310224+0000","last_peered":"2024-09-19T05:41:13.310224+0000","last_clean":"2024-09-19T05:41:13.310224+0000","last_became_active":"2024-09-19T05:38:59.857271+0000","last_became_peered":"2024-09-19T05:38:59.857271+0000","last_unstale":"2024-09-19T05:41:13.310224+0000","last_undegraded":"2024-09-19T05:41:13.310224+0000","last_fullsized":"2024-09-19T05:41:13.310224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:51.193860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195002+0000","last_change":"2024-09-19T05:39:08.950016+0000","last_active":"2024-09-19T05:41:11.195002+0000","last_peered":"2024-09-19T05:41:11.195002+0000","last_clean":"2024-09-19T05:41:11.195002+0000","last_became_active":"2024-09-19T05:39:08.949264+0000","last_became_peered":"2024-09-19T05:39:08.949264+0000","last_unstale":"2024-09-19T05:41:11.195002+0000","last_undegraded":"2024-09-19T05:41:11.195002+0000","last_fullsized":"2024-09-19T05:41:11.195002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:56.194697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197505+0000","last_change":"2024-09-19T05:39:01.900954+0000","last_active":"2024-09-19T05:41:11.197505+0000","last_peered":"2024-09-19T05:41:11.197505+0000","last_clean":"2024-09-19T05:41:11.197505+0000","last_became_active":"2024-09-19T05:39:01.900608+0000","last_became_peered":"2024-09-19T05:39:01.900608+0000","last_unstale":"2024-09-19T05:41:11.197505+0000","last_undegraded":"2024-09-19T05:41:11.197505+0000","last_fullsized":"2024-09-19T05:41:11.197505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:32.238837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208944+0000","last_change":"2024-09-19T05:38:59.858505+0000","last_active":"2024-09-19T05:41:11.208944+0000","last_peered":"2024-09-19T05:41:11.208944+0000","last_clean":"2024-09-19T05:41:11.208944+0000","last_became_active":"2024-09-19T05:38:59.858157+0000","last_became_peered":"2024-09-19T05:38:59.858157+0000","last_unstale":"2024-09-19T05:41:11.208944+0000","last_undegraded":"2024-09-19T05:41:11.208944+0000","last_fullsized":"2024-09-19T05:41:11.208944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:30.483603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199696+0000","last_change":"2024-09-19T05:39:08.957136+0000","last_active":"2024-09-19T05:41:11.199696+0000","last_peered":"2024-09-19T05:41:11.199696+0000","last_clean":"2024-09-19T05:41:11.199696+0000","last_became_active":"2024-09-19T05:39:08.956732+0000","last_became_peered":"2024-09-19T05:39:08.956732+0000","last_unstale":"2024-09-19T05:41:11.199696+0000","last_undegraded":"2024-09-19T05:41:11.199696+0000","last_fullsized":"2024-09-19T05:41:11.199696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:13.160044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199690+0000","last_change":"2024-09-19T05:39:01.896877+0000","last_active":"2024-09-19T05:41:11.199690+0000","last_peered":"2024-09-19T05:41:11.199690+0000","last_clean":"2024-09-19T05:41:11.199690+0000","last_became_active":"2024-09-19T05:39:01.896186+0000","last_became_peered":"2024-09-19T05:39:01.896186+0000","last_unstale":"2024-09-19T05:41:11.199690+0000","last_undegraded":"2024-09-19T05:41:11.199690+0000","last_fullsized":"2024-09-19T05:41:11.199690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:01.565126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209598+0000","last_change":"2024-09-19T05:39:00.095805+0000","last_active":"2024-09-19T05:41:11.209598+0000","last_peered":"2024-09-19T05:41:11.209598+0000","last_clean":"2024-09-19T05:41:11.209598+0000","last_became_active":"2024-09-19T05:39:00.095408+0000","last_became_peered":"2024-09-19T05:39:00.095408+0000","last_unstale":"2024-09-19T05:41:11.209598+0000","last_undegraded":"2024-09-19T05:41:11.209598+0000","last_fullsized":"2024-09-19T05:41:11.209598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:13:30.706068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195060+0000","last_change":"2024-09-19T05:39:09.049541+0000","last_active":"2024-09-19T05:41:11.195060+0000","last_peered":"2024-09-19T05:41:11.195060+0000","last_clean":"2024-09-19T05:41:11.195060+0000","last_became_active":"2024-09-19T05:39:09.049102+0000","last_became_peered":"2024-09-19T05:39:09.049102+0000","last_unstale":"2024-09-19T05:41:11.195060+0000","last_undegraded":"2024-09-19T05:41:11.195060+0000","last_fullsized":"2024-09-19T05:41:11.195060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:22.701759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198875+0000","last_change":"2024-09-19T05:39:01.891073+0000","last_active":"2024-09-19T05:41:11.198875+0000","last_peered":"2024-09-19T05:41:11.198875+0000","last_clean":"2024-09-19T05:41:11.198875+0000","last_became_active":"2024-09-19T05:39:01.890713+0000","last_became_peered":"2024-09-19T05:39:01.890713+0000","last_unstale":"2024-09-19T05:41:11.198875+0000","last_undegraded":"2024-09-19T05:41:11.198875+0000","last_fullsized":"2024-09-19T05:41:11.198875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:59.298502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195267+0000","last_change":"2024-09-19T05:38:59.969199+0000","last_active":"2024-09-19T05:41:11.195267+0000","last_peered":"2024-09-19T05:41:11.195267+0000","last_clean":"2024-09-19T05:41:11.195267+0000","last_became_active":"2024-09-19T05:38:59.968875+0000","last_became_peered":"2024-09-19T05:38:59.968875+0000","last_unstale":"2024-09-19T05:41:11.195267+0000","last_undegraded":"2024-09-19T05:41:11.195267+0000","last_fullsized":"2024-09-19T05:41:11.195267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:31.662581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195400+0000","last_change":"2024-09-19T05:39:08.945694+0000","last_active":"2024-09-19T05:41:11.195400+0000","last_peered":"2024-09-19T05:41:11.195400+0000","last_clean":"2024-09-19T05:41:11.195400+0000","last_became_active":"2024-09-19T05:39:08.945262+0000","last_became_peered":"2024-09-19T05:39:08.945262+0000","last_unstale":"2024-09-19T05:41:11.195400+0000","last_undegraded":"2024-09-19T05:41:11.195400+0000","last_fullsized":"2024-09-19T05:41:11.195400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:33.528819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208869+0000","last_change":"2024-09-19T05:39:01.899873+0000","last_active":"2024-09-19T05:41:11.208869+0000","last_peered":"2024-09-19T05:41:11.208869+0000","last_clean":"2024-09-19T05:41:11.208869+0000","last_became_active":"2024-09-19T05:39:01.899566+0000","last_became_peered":"2024-09-19T05:39:01.899566+0000","last_unstale":"2024-09-19T05:41:11.208869+0000","last_undegraded":"2024-09-19T05:41:11.208869+0000","last_fullsized":"2024-09-19T05:41:11.208869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:11.188289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.454620+0000","last_change":"2024-09-19T05:38:59.850881+0000","last_active":"2024-09-19T05:41:11.454620+0000","last_peered":"2024-09-19T05:41:11.454620+0000","last_clean":"2024-09-19T05:41:11.454620+0000","last_became_active":"2024-09-19T05:38:59.850542+0000","last_became_peered":"2024-09-19T05:38:59.850542+0000","last_unstale":"2024-09-19T05:41:11.454620+0000","last_undegraded":"2024-09-19T05:41:11.454620+0000","last_fullsized":"2024-09-19T05:41:11.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:00.393569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208319+0000","last_change":"2024-09-19T05:39:09.048832+0000","last_active":"2024-09-19T05:41:11.208319+0000","last_peered":"2024-09-19T05:41:11.208319+0000","last_clean":"2024-09-19T05:41:11.208319+0000","last_became_active":"2024-09-19T05:39:09.047888+0000","last_became_peered":"2024-09-19T05:39:09.047888+0000","last_unstale":"2024-09-19T05:41:11.208319+0000","last_undegraded":"2024-09-19T05:41:11.208319+0000","last_fullsized":"2024-09-19T05:41:11.208319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:39.718391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.893427+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.893116+0000","last_became_peered":"2024-09-19T05:39:01.893116+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:16:47.033664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.211599+0000","last_change":"2024-09-19T05:41:11.210967+0000","last_active":"2024-09-19T05:41:11.211599+0000","last_peered":"2024-09-19T05:41:11.211599+0000","last_clean":"2024-09-19T05:41:11.211599+0000","last_became_active":"2024-09-19T05:41:11.201065+0000","last_became_peered":"2024-09-19T05:41:11.201065+0000","last_unstale":"2024-09-19T05:41:11.211599+0000","last_undegraded":"2024-09-19T05:41:11.211599+0000","last_fullsized":"2024-09-19T05:41:11.211599+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:54.393420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197544+0000","last_change":"2024-09-19T05:39:08.950553+0000","last_active":"2024-09-19T05:41:11.197544+0000","last_peered":"2024-09-19T05:41:11.197544+0000","last_clean":"2024-09-19T05:41:11.197544+0000","last_became_active":"2024-09-19T05:39:08.950139+0000","last_became_peered":"2024-09-19T05:39:08.950139+0000","last_unstale":"2024-09-19T05:41:11.197544+0000","last_undegraded":"2024-09-19T05:41:11.197544+0000","last_fullsized":"2024-09-19T05:41:11.197544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:45.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209065+0000","last_change":"2024-09-19T05:39:01.894749+0000","last_active":"2024-09-19T05:41:11.209065+0000","last_peered":"2024-09-19T05:41:11.209065+0000","last_clean":"2024-09-19T05:41:11.209065+0000","last_became_active":"2024-09-19T05:39:01.894439+0000","last_became_peered":"2024-09-19T05:39:01.894439+0000","last_unstale":"2024-09-19T05:41:11.209065+0000","last_undegraded":"2024-09-19T05:41:11.209065+0000","last_fullsized":"2024-09-19T05:41:11.209065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:38.440348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199469+0000","last_change":"2024-09-19T05:39:00.096325+0000","last_active":"2024-09-19T05:41:11.199469+0000","last_peered":"2024-09-19T05:41:11.199469+0000","last_clean":"2024-09-19T05:41:11.199469+0000","last_became_active":"2024-09-19T05:39:00.095690+0000","last_became_peered":"2024-09-19T05:39:00.095690+0000","last_unstale":"2024-09-19T05:41:11.199469+0000","last_undegraded":"2024-09-19T05:41:11.199469+0000","last_fullsized":"2024-09-19T05:41:11.199469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:19.008741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199830+0000","last_change":"2024-09-19T05:39:09.048863+0000","last_active":"2024-09-19T05:41:11.199830+0000","last_peered":"2024-09-19T05:41:11.199830+0000","last_clean":"2024-09-19T05:41:11.199830+0000","last_became_active":"2024-09-19T05:39:09.048337+0000","last_became_peered":"2024-09-19T05:39:09.048337+0000","last_unstale":"2024-09-19T05:41:11.199830+0000","last_undegraded":"2024-09-19T05:41:11.199830+0000","last_fullsized":"2024-09-19T05:41:11.199830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:07.779959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209286+0000","last_change":"2024-09-19T05:39:01.901409+0000","last_active":"2024-09-19T05:41:11.209286+0000","last_peered":"2024-09-19T05:41:11.209286+0000","last_clean":"2024-09-19T05:41:11.209286+0000","last_became_active":"2024-09-19T05:39:01.899820+0000","last_became_peered":"2024-09-19T05:39:01.899820+0000","last_unstale":"2024-09-19T05:41:11.209286+0000","last_undegraded":"2024-09-19T05:41:11.209286+0000","last_fullsized":"2024-09-19T05:41:11.209286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:30.505960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:33.257170+0000","last_change":"2024-09-19T05:38:59.854238+0000","last_active":"2024-09-19T05:41:33.257170+0000","last_peered":"2024-09-19T05:41:33.257170+0000","last_clean":"2024-09-19T05:41:33.257170+0000","last_became_active":"2024-09-19T05:38:59.853168+0000","last_became_peered":"2024-09-19T05:38:59.853168+0000","last_unstale":"2024-09-19T05:41:33.257170+0000","last_undegraded":"2024-09-19T05:41:33.257170+0000","last_fullsized":"2024-09-19T05:41:33.257170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:39.205775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209476+0000","last_change":"2024-09-19T05:39:08.945532+0000","last_active":"2024-09-19T05:41:11.209476+0000","last_peered":"2024-09-19T05:41:11.209476+0000","last_clean":"2024-09-19T05:41:11.209476+0000","last_became_active":"2024-09-19T05:39:08.945171+0000","last_became_peered":"2024-09-19T05:39:08.945171+0000","last_unstale":"2024-09-19T05:41:11.209476+0000","last_undegraded":"2024-09-19T05:41:11.209476+0000","last_fullsized":"2024-09-19T05:41:11.209476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:02.867312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208429+0000","last_change":"2024-09-19T05:40:10.594007+0000","last_active":"2024-09-19T05:41:11.208429+0000","last_peered":"2024-09-19T05:41:11.208429+0000","last_clean":"2024-09-19T05:41:11.208429+0000","last_became_active":"2024-09-19T05:40:10.593493+0000","last_became_peered":"2024-09-19T05:40:10.593493+0000","last_unstale":"2024-09-19T05:41:11.208429+0000","last_undegraded":"2024-09-19T05:41:11.208429+0000","last_fullsized":"2024-09-19T05:41:11.208429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:58.225840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:33.256492+0000","last_change":"2024-09-19T05:39:00.095851+0000","last_active":"2024-09-19T05:41:33.256492+0000","last_peered":"2024-09-19T05:41:33.256492+0000","last_clean":"2024-09-19T05:41:33.256492+0000","last_became_active":"2024-09-19T05:39:00.095527+0000","last_became_peered":"2024-09-19T05:39:00.095527+0000","last_unstale":"2024-09-19T05:41:33.256492+0000","last_undegraded":"2024-09-19T05:41:33.256492+0000","last_fullsized":"2024-09-19T05:41:33.256492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:26.353613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209540+0000","last_change":"2024-09-19T05:39:09.048330+0000","last_active":"2024-09-19T05:41:11.209540+0000","last_peered":"2024-09-19T05:41:11.209540+0000","last_clean":"2024-09-19T05:41:11.209540+0000","last_became_active":"2024-09-19T05:39:09.047899+0000","last_became_peered":"2024-09-19T05:39:09.047899+0000","last_unstale":"2024-09-19T05:41:11.209540+0000","last_undegraded":"2024-09-19T05:41:11.209540+0000","last_fullsized":"2024-09-19T05:41:11.209540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:46.903138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208853+0000","last_change":"2024-09-19T05:39:01.892311+0000","last_active":"2024-09-19T05:41:11.208853+0000","last_peered":"2024-09-19T05:41:11.208853+0000","last_clean":"2024-09-19T05:41:11.208853+0000","last_became_active":"2024-09-19T05:39:01.892006+0000","last_became_peered":"2024-09-19T05:39:01.892006+0000","last_unstale":"2024-09-19T05:41:11.208853+0000","last_undegraded":"2024-09-19T05:41:11.208853+0000","last_fullsized":"2024-09-19T05:41:11.208853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:18:58.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199656+0000","last_change":"2024-09-19T05:38:59.851454+0000","last_active":"2024-09-19T05:41:11.199656+0000","last_peered":"2024-09-19T05:41:11.199656+0000","last_clean":"2024-09-19T05:41:11.199656+0000","last_became_active":"2024-09-19T05:38:59.851088+0000","last_became_peered":"2024-09-19T05:38:59.851088+0000","last_unstale":"2024-09-19T05:41:11.199656+0000","last_undegraded":"2024-09-19T05:41:11.199656+0000","last_fullsized":"2024-09-19T05:41:11.199656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:16.063648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209029+0000","last_change":"2024-09-19T05:39:09.049926+0000","last_active":"2024-09-19T05:41:11.209029+0000","last_peered":"2024-09-19T05:41:11.209029+0000","last_clean":"2024-09-19T05:41:11.209029+0000","last_became_active":"2024-09-19T05:39:09.049131+0000","last_became_peered":"2024-09-19T05:39:09.049131+0000","last_unstale":"2024-09-19T05:41:11.209029+0000","last_undegraded":"2024-09-19T05:41:11.209029+0000","last_fullsized":"2024-09-19T05:41:11.209029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:54.154774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208986+0000","last_change":"2024-09-19T05:39:01.897552+0000","last_active":"2024-09-19T05:41:11.208986+0000","last_peered":"2024-09-19T05:41:11.208986+0000","last_clean":"2024-09-19T05:41:11.208986+0000","last_became_active":"2024-09-19T05:39:01.897142+0000","last_became_peered":"2024-09-19T05:39:01.897142+0000","last_unstale":"2024-09-19T05:41:11.208986+0000","last_undegraded":"2024-09-19T05:41:11.208986+0000","last_fullsized":"2024-09-19T05:41:11.208986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:25.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195015+0000","last_change":"2024-09-19T05:38:59.856582+0000","last_active":"2024-09-19T05:41:11.195015+0000","last_peered":"2024-09-19T05:41:11.195015+0000","last_clean":"2024-09-19T05:41:11.195015+0000","last_became_active":"2024-09-19T05:38:59.856162+0000","last_became_peered":"2024-09-19T05:38:59.856162+0000","last_unstale":"2024-09-19T05:41:11.195015+0000","last_undegraded":"2024-09-19T05:41:11.195015+0000","last_fullsized":"2024-09-19T05:41:11.195015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:29.685471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199828+0000","last_change":"2024-09-19T05:37:49.568246+0000","last_active":"2024-09-19T05:41:11.199828+0000","last_peered":"2024-09-19T05:41:11.199828+0000","last_clean":"2024-09-19T05:41:11.199828+0000","last_became_active":"2024-09-19T05:37:49.567761+0000","last_became_peered":"2024-09-19T05:37:49.567761+0000","last_unstale":"2024-09-19T05:41:11.199828+0000","last_undegraded":"2024-09-19T05:41:11.199828+0000","last_fullsized":"2024-09-19T05:41:11.199828+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:34:24.193602+0000","last_clean_scrub_stamp":"2024-09-19T05:34:24.193602+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:28:22.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200179+0000","last_change":"2024-09-19T05:39:08.952911+0000","last_active":"2024-09-19T05:41:11.200179+0000","last_peered":"2024-09-19T05:41:11.200179+0000","last_clean":"2024-09-19T05:41:11.200179+0000","last_became_active":"2024-09-19T05:39:08.952456+0000","last_became_peered":"2024-09-19T05:39:08.952456+0000","last_unstale":"2024-09-19T05:41:11.200179+0000","last_undegraded":"2024-09-19T05:41:11.200179+0000","last_fullsized":"2024-09-19T05:41:11.200179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200398+0000","last_change":"2024-09-19T05:39:01.896775+0000","last_active":"2024-09-19T05:41:11.200398+0000","last_peered":"2024-09-19T05:41:11.200398+0000","last_clean":"2024-09-19T05:41:11.200398+0000","last_became_active":"2024-09-19T05:39:01.896451+0000","last_became_peered":"2024-09-19T05:39:01.896451+0000","last_unstale":"2024-09-19T05:41:11.200398+0000","last_undegraded":"2024-09-19T05:41:11.200398+0000","last_fullsized":"2024-09-19T05:41:11.200398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:19.596767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.237830+0000","last_change":"2024-09-19T05:41:11.205122+0000","last_active":"2024-09-19T05:41:32.237830+0000","last_peered":"2024-09-19T05:41:32.237830+0000","last_clean":"2024-09-19T05:41:32.237830+0000","last_became_active":"2024-09-19T05:41:11.200544+0000","last_became_peered":"2024-09-19T05:41:11.200544+0000","last_unstale":"2024-09-19T05:41:32.237830+0000","last_undegraded":"2024-09-19T05:41:32.237830+0000","last_fullsized":"2024-09-19T05:41:32.237830+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:11.609857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2012,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209616+0000","last_change":"2024-09-19T05:40:10.596589+0000","last_active":"2024-09-19T05:41:11.209616+0000","last_peered":"2024-09-19T05:41:11.209616+0000","last_clean":"2024-09-19T05:41:11.209616+0000","last_became_active":"2024-09-19T05:40:10.596087+0000","last_became_peered":"2024-09-19T05:40:10.596087+0000","last_unstale":"2024-09-19T05:41:11.209616+0000","last_undegraded":"2024-09-19T05:41:11.209616+0000","last_fullsized":"2024-09-19T05:41:11.209616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200123+0000","last_change":"2024-09-19T05:40:10.594372+0000","last_active":"2024-09-19T05:41:11.200123+0000","last_peered":"2024-09-19T05:41:11.200123+0000","last_clean":"2024-09-19T05:41:11.200123+0000","last_became_active":"2024-09-19T05:40:10.594018+0000","last_became_peered":"2024-09-19T05:40:10.594018+0000","last_unstale":"2024-09-19T05:41:11.200123+0000","last_undegraded":"2024-09-19T05:41:11.200123+0000","last_fullsized":"2024-09-19T05:41:11.200123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:23.706247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209269+0000","last_change":"2024-09-19T05:38:59.859421+0000","last_active":"2024-09-19T05:41:11.209269+0000","last_peered":"2024-09-19T05:41:11.209269+0000","last_clean":"2024-09-19T05:41:11.209269+0000","last_became_active":"2024-09-19T05:38:59.859090+0000","last_became_peered":"2024-09-19T05:38:59.859090+0000","last_unstale":"2024-09-19T05:41:11.209269+0000","last_undegraded":"2024-09-19T05:41:11.209269+0000","last_fullsized":"2024-09-19T05:41:11.209269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:33.279407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198850+0000","last_change":"2024-09-19T05:39:08.949115+0000","last_active":"2024-09-19T05:41:11.198850+0000","last_peered":"2024-09-19T05:41:11.198850+0000","last_clean":"2024-09-19T05:41:11.198850+0000","last_became_active":"2024-09-19T05:39:08.948699+0000","last_became_peered":"2024-09-19T05:39:08.948699+0000","last_unstale":"2024-09-19T05:41:11.198850+0000","last_undegraded":"2024-09-19T05:41:11.198850+0000","last_fullsized":"2024-09-19T05:41:11.198850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:59.917407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200345+0000","last_change":"2024-09-19T05:39:01.901079+0000","last_active":"2024-09-19T05:41:11.200345+0000","last_peered":"2024-09-19T05:41:11.200345+0000","last_clean":"2024-09-19T05:41:11.200345+0000","last_became_active":"2024-09-19T05:39:01.900739+0000","last_became_peered":"2024-09-19T05:39:01.900739+0000","last_unstale":"2024-09-19T05:41:11.200345+0000","last_undegraded":"2024-09-19T05:41:11.200345+0000","last_fullsized":"2024-09-19T05:41:11.200345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:13.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209419+0000","last_change":"2024-09-19T05:38:59.854762+0000","last_active":"2024-09-19T05:41:11.209419+0000","last_peered":"2024-09-19T05:41:11.209419+0000","last_clean":"2024-09-19T05:41:11.209419+0000","last_became_active":"2024-09-19T05:38:59.854393+0000","last_became_peered":"2024-09-19T05:38:59.854393+0000","last_unstale":"2024-09-19T05:41:11.209419+0000","last_undegraded":"2024-09-19T05:41:11.209419+0000","last_fullsized":"2024-09-19T05:41:11.209419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:18.582009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208290+0000","last_change":"2024-09-19T05:39:08.954283+0000","last_active":"2024-09-19T05:41:11.208290+0000","last_peered":"2024-09-19T05:41:11.208290+0000","last_clean":"2024-09-19T05:41:11.208290+0000","last_became_active":"2024-09-19T05:39:08.953779+0000","last_became_peered":"2024-09-19T05:39:08.953779+0000","last_unstale":"2024-09-19T05:41:11.208290+0000","last_undegraded":"2024-09-19T05:41:11.208290+0000","last_fullsized":"2024-09-19T05:41:11.208290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:38.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197736+0000","last_change":"2024-09-19T05:39:01.900599+0000","last_active":"2024-09-19T05:41:11.197736+0000","last_peered":"2024-09-19T05:41:11.197736+0000","last_clean":"2024-09-19T05:41:11.197736+0000","last_became_active":"2024-09-19T05:39:01.899848+0000","last_became_peered":"2024-09-19T05:39:01.899848+0000","last_unstale":"2024-09-19T05:41:11.197736+0000","last_undegraded":"2024-09-19T05:41:11.197736+0000","last_fullsized":"2024-09-19T05:41:11.197736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:52.631828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200434+0000","last_change":"2024-09-19T05:38:59.853437+0000","last_active":"2024-09-19T05:41:11.200434+0000","last_peered":"2024-09-19T05:41:11.200434+0000","last_clean":"2024-09-19T05:41:11.200434+0000","last_became_active":"2024-09-19T05:38:59.852909+0000","last_became_peered":"2024-09-19T05:38:59.852909+0000","last_unstale":"2024-09-19T05:41:11.200434+0000","last_undegraded":"2024-09-19T05:41:11.200434+0000","last_fullsized":"2024-09-19T05:41:11.200434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:25.284152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208357+0000","last_change":"2024-09-19T05:39:08.951646+0000","last_active":"2024-09-19T05:41:11.208357+0000","last_peered":"2024-09-19T05:41:11.208357+0000","last_clean":"2024-09-19T05:41:11.208357+0000","last_became_active":"2024-09-19T05:39:08.951140+0000","last_became_peered":"2024-09-19T05:39:08.951140+0000","last_unstale":"2024-09-19T05:41:11.208357+0000","last_undegraded":"2024-09-19T05:41:11.208357+0000","last_fullsized":"2024-09-19T05:41:11.208357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:16.321134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208646+0000","last_change":"2024-09-19T05:40:10.596504+0000","last_active":"2024-09-19T05:41:11.208646+0000","last_peered":"2024-09-19T05:41:11.208646+0000","last_clean":"2024-09-19T05:41:11.208646+0000","last_became_active":"2024-09-19T05:40:10.596112+0000","last_became_peered":"2024-09-19T05:40:10.596112+0000","last_unstale":"2024-09-19T05:41:11.208646+0000","last_undegraded":"2024-09-19T05:41:11.208646+0000","last_fullsized":"2024-09-19T05:41:11.208646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:02.818567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199743+0000","last_change":"2024-09-19T05:39:00.095964+0000","last_active":"2024-09-19T05:41:11.199743+0000","last_peered":"2024-09-19T05:41:11.199743+0000","last_clean":"2024-09-19T05:41:11.199743+0000","last_became_active":"2024-09-19T05:39:00.095543+0000","last_became_peered":"2024-09-19T05:39:00.095543+0000","last_unstale":"2024-09-19T05:41:11.199743+0000","last_undegraded":"2024-09-19T05:41:11.199743+0000","last_fullsized":"2024-09-19T05:41:11.199743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:48.693639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209288+0000","last_change":"2024-09-19T05:39:08.949007+0000","last_active":"2024-09-19T05:41:11.209288+0000","last_peered":"2024-09-19T05:41:11.209288+0000","last_clean":"2024-09-19T05:41:11.209288+0000","last_became_active":"2024-09-19T05:39:08.948612+0000","last_became_peered":"2024-09-19T05:39:08.948612+0000","last_unstale":"2024-09-19T05:41:11.209288+0000","last_undegraded":"2024-09-19T05:41:11.209288+0000","last_fullsized":"2024-09-19T05:41:11.209288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:44.295422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199739+0000","last_change":"2024-09-19T05:39:01.898813+0000","last_active":"2024-09-19T05:41:11.199739+0000","last_peered":"2024-09-19T05:41:11.199739+0000","last_clean":"2024-09-19T05:41:11.199739+0000","last_became_active":"2024-09-19T05:39:01.898511+0000","last_became_peered":"2024-09-19T05:39:01.898511+0000","last_unstale":"2024-09-19T05:41:11.199739+0000","last_undegraded":"2024-09-19T05:41:11.199739+0000","last_fullsized":"2024-09-19T05:41:11.199739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:20.553036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.255893+0000","last_change":"2024-09-19T05:38:59.856433+0000","last_active":"2024-09-19T05:41:32.255893+0000","last_peered":"2024-09-19T05:41:32.255893+0000","last_clean":"2024-09-19T05:41:32.255893+0000","last_became_active":"2024-09-19T05:38:59.856087+0000","last_became_peered":"2024-09-19T05:38:59.856087+0000","last_unstale":"2024-09-19T05:41:32.255893+0000","last_undegraded":"2024-09-19T05:41:32.255893+0000","last_fullsized":"2024-09-19T05:41:32.255893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:51:17.411552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199669+0000","last_change":"2024-09-19T05:39:08.955520+0000","last_active":"2024-09-19T05:41:11.199669+0000","last_peered":"2024-09-19T05:41:11.199669+0000","last_clean":"2024-09-19T05:41:11.199669+0000","last_became_active":"2024-09-19T05:39:08.955119+0000","last_became_peered":"2024-09-19T05:39:08.955119+0000","last_unstale":"2024-09-19T05:41:11.199669+0000","last_undegraded":"2024-09-19T05:41:11.199669+0000","last_fullsized":"2024-09-19T05:41:11.199669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:44.491396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199990+0000","last_change":"2024-09-19T05:39:01.895306+0000","last_active":"2024-09-19T05:41:11.199990+0000","last_peered":"2024-09-19T05:41:11.199990+0000","last_clean":"2024-09-19T05:41:11.199990+0000","last_became_active":"2024-09-19T05:39:01.894893+0000","last_became_peered":"2024-09-19T05:39:01.894893+0000","last_unstale":"2024-09-19T05:41:11.199990+0000","last_undegraded":"2024-09-19T05:41:11.199990+0000","last_fullsized":"2024-09-19T05:41:11.199990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:59.834021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208852+0000","last_change":"2024-09-19T05:38:59.855025+0000","last_active":"2024-09-19T05:41:11.208852+0000","last_peered":"2024-09-19T05:41:11.208852+0000","last_clean":"2024-09-19T05:41:11.208852+0000","last_became_active":"2024-09-19T05:38:59.854664+0000","last_became_peered":"2024-09-19T05:38:59.854664+0000","last_unstale":"2024-09-19T05:41:11.208852+0000","last_undegraded":"2024-09-19T05:41:11.208852+0000","last_fullsized":"2024-09-19T05:41:11.208852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:38.148713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197463+0000","last_change":"2024-09-19T05:39:08.949258+0000","last_active":"2024-09-19T05:41:11.197463+0000","last_peered":"2024-09-19T05:41:11.197463+0000","last_clean":"2024-09-19T05:41:11.197463+0000","last_became_active":"2024-09-19T05:39:08.948624+0000","last_became_peered":"2024-09-19T05:39:08.948624+0000","last_unstale":"2024-09-19T05:41:11.197463+0000","last_undegraded":"2024-09-19T05:41:11.197463+0000","last_fullsized":"2024-09-19T05:41:11.197463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.752512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209172+0000","last_change":"2024-09-19T05:39:01.895486+0000","last_active":"2024-09-19T05:41:11.209172+0000","last_peered":"2024-09-19T05:41:11.209172+0000","last_clean":"2024-09-19T05:41:11.209172+0000","last_became_active":"2024-09-19T05:39:01.895182+0000","last_became_peered":"2024-09-19T05:39:01.895182+0000","last_unstale":"2024-09-19T05:41:11.209172+0000","last_undegraded":"2024-09-19T05:41:11.209172+0000","last_fullsized":"2024-09-19T05:41:11.209172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:15.175938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195638+0000","last_change":"2024-09-19T05:38:59.860071+0000","last_active":"2024-09-19T05:41:11.195638+0000","last_peered":"2024-09-19T05:41:11.195638+0000","last_clean":"2024-09-19T05:41:11.195638+0000","last_became_active":"2024-09-19T05:38:59.859768+0000","last_became_peered":"2024-09-19T05:38:59.859768+0000","last_unstale":"2024-09-19T05:41:11.195638+0000","last_undegraded":"2024-09-19T05:41:11.195638+0000","last_fullsized":"2024-09-19T05:41:11.195638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:34.797883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208925+0000","last_change":"2024-09-19T05:39:08.953136+0000","last_active":"2024-09-19T05:41:11.208925+0000","last_peered":"2024-09-19T05:41:11.208925+0000","last_clean":"2024-09-19T05:41:11.208925+0000","last_became_active":"2024-09-19T05:39:08.952415+0000","last_became_peered":"2024-09-19T05:39:08.952415+0000","last_unstale":"2024-09-19T05:41:11.208925+0000","last_undegraded":"2024-09-19T05:41:11.208925+0000","last_fullsized":"2024-09-19T05:41:11.208925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:49:26.334648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200325+0000","last_change":"2024-09-19T05:39:01.898623+0000","last_active":"2024-09-19T05:41:11.200325+0000","last_peered":"2024-09-19T05:41:11.200325+0000","last_clean":"2024-09-19T05:41:11.200325+0000","last_became_active":"2024-09-19T05:39:01.898285+0000","last_became_peered":"2024-09-19T05:39:01.898285+0000","last_unstale":"2024-09-19T05:41:11.200325+0000","last_undegraded":"2024-09-19T05:41:11.200325+0000","last_fullsized":"2024-09-19T05:41:11.200325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:25.302630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.259718+0000","last_change":"2024-09-19T05:38:59.859729+0000","last_active":"2024-09-19T05:41:31.259718+0000","last_peered":"2024-09-19T05:41:31.259718+0000","last_clean":"2024-09-19T05:41:31.259718+0000","last_became_active":"2024-09-19T05:38:59.859089+0000","last_became_peered":"2024-09-19T05:38:59.859089+0000","last_unstale":"2024-09-19T05:41:31.259718+0000","last_undegraded":"2024-09-19T05:41:31.259718+0000","last_fullsized":"2024-09-19T05:41:31.259718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:51.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65179,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198935+0000","last_change":"2024-09-19T05:39:08.954203+0000","last_active":"2024-09-19T05:41:11.198935+0000","last_peered":"2024-09-19T05:41:11.198935+0000","last_clean":"2024-09-19T05:41:11.198935+0000","last_became_active":"2024-09-19T05:39:08.952814+0000","last_became_peered":"2024-09-19T05:39:08.952814+0000","last_unstale":"2024-09-19T05:41:11.198935+0000","last_undegraded":"2024-09-19T05:41:11.198935+0000","last_fullsized":"2024-09-19T05:41:11.198935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:10.847380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210003+0000","last_change":"2024-09-19T05:39:01.894636+0000","last_active":"2024-09-19T05:41:11.210003+0000","last_peered":"2024-09-19T05:41:11.210003+0000","last_clean":"2024-09-19T05:41:11.210003+0000","last_became_active":"2024-09-19T05:39:01.894306+0000","last_became_peered":"2024-09-19T05:39:01.894306+0000","last_unstale":"2024-09-19T05:41:11.210003+0000","last_undegraded":"2024-09-19T05:41:11.210003+0000","last_fullsized":"2024-09-19T05:41:11.210003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:17.325493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209213+0000","last_change":"2024-09-19T05:39:00.097987+0000","last_active":"2024-09-19T05:41:11.209213+0000","last_peered":"2024-09-19T05:41:11.209213+0000","last_clean":"2024-09-19T05:41:11.209213+0000","last_became_active":"2024-09-19T05:39:00.097622+0000","last_became_peered":"2024-09-19T05:39:00.097622+0000","last_unstale":"2024-09-19T05:41:11.209213+0000","last_undegraded":"2024-09-19T05:41:11.209213+0000","last_fullsized":"2024-09-19T05:41:11.209213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:36.219743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.194832+0000","last_change":"2024-09-19T05:39:08.949549+0000","last_active":"2024-09-19T05:41:11.194832+0000","last_peered":"2024-09-19T05:41:11.194832+0000","last_clean":"2024-09-19T05:41:11.194832+0000","last_became_active":"2024-09-19T05:39:08.949115+0000","last_became_peered":"2024-09-19T05:39:08.949115+0000","last_unstale":"2024-09-19T05:41:11.194832+0000","last_undegraded":"2024-09-19T05:41:11.194832+0000","last_fullsized":"2024-09-19T05:41:11.194832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:34.950351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199324+0000","last_change":"2024-09-19T05:39:01.901972+0000","last_active":"2024-09-19T05:41:11.199324+0000","last_peered":"2024-09-19T05:41:11.199324+0000","last_clean":"2024-09-19T05:41:11.199324+0000","last_became_active":"2024-09-19T05:39:01.901661+0000","last_became_peered":"2024-09-19T05:39:01.901661+0000","last_unstale":"2024-09-19T05:41:11.199324+0000","last_undegraded":"2024-09-19T05:41:11.199324+0000","last_fullsized":"2024-09-19T05:41:11.199324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:09:49.029844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200329+0000","last_change":"2024-09-19T05:38:59.855058+0000","last_active":"2024-09-19T05:41:11.200329+0000","last_peered":"2024-09-19T05:41:11.200329+0000","last_clean":"2024-09-19T05:41:11.200329+0000","last_became_active":"2024-09-19T05:38:59.854669+0000","last_became_peered":"2024-09-19T05:38:59.854669+0000","last_unstale":"2024-09-19T05:41:11.200329+0000","last_undegraded":"2024-09-19T05:41:11.200329+0000","last_fullsized":"2024-09-19T05:41:11.200329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:46.388879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208882+0000","last_change":"2024-09-19T05:40:10.893405+0000","last_active":"2024-09-19T05:41:11.208882+0000","last_peered":"2024-09-19T05:41:11.208882+0000","last_clean":"2024-09-19T05:41:11.208882+0000","last_became_active":"2024-09-19T05:40:10.892932+0000","last_became_peered":"2024-09-19T05:40:10.892932+0000","last_unstale":"2024-09-19T05:41:11.208882+0000","last_undegraded":"2024-09-19T05:41:11.208882+0000","last_fullsized":"2024-09-19T05:41:11.208882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:56.153847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210119+0000","last_change":"2024-09-19T05:39:01.894997+0000","last_active":"2024-09-19T05:41:11.210119+0000","last_peered":"2024-09-19T05:41:11.210119+0000","last_clean":"2024-09-19T05:41:11.210119+0000","last_became_active":"2024-09-19T05:39:01.893091+0000","last_became_peered":"2024-09-19T05:39:01.893091+0000","last_unstale":"2024-09-19T05:41:11.210119+0000","last_undegraded":"2024-09-19T05:41:11.210119+0000","last_fullsized":"2024-09-19T05:41:11.210119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:09.918566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:31.235843+0000","last_change":"2024-09-19T05:38:59.859314+0000","last_active":"2024-09-19T05:41:31.235843+0000","last_peered":"2024-09-19T05:41:31.235843+0000","last_clean":"2024-09-19T05:41:31.235843+0000","last_became_active":"2024-09-19T05:38:59.858932+0000","last_became_peered":"2024-09-19T05:38:59.858932+0000","last_unstale":"2024-09-19T05:41:31.235843+0000","last_undegraded":"2024-09-19T05:41:31.235843+0000","last_fullsized":"2024-09-19T05:41:31.235843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:51.747916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208507+0000","last_change":"2024-09-19T05:39:08.955164+0000","last_active":"2024-09-19T05:41:11.208507+0000","last_peered":"2024-09-19T05:41:11.208507+0000","last_clean":"2024-09-19T05:41:11.208507+0000","last_became_active":"2024-09-19T05:39:08.954784+0000","last_became_peered":"2024-09-19T05:39:08.954784+0000","last_unstale":"2024-09-19T05:41:11.208507+0000","last_undegraded":"2024-09-19T05:41:11.208507+0000","last_fullsized":"2024-09-19T05:41:11.208507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:20.215846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198681+0000","last_change":"2024-09-19T05:39:08.949123+0000","last_active":"2024-09-19T05:41:11.198681+0000","last_peered":"2024-09-19T05:41:11.198681+0000","last_clean":"2024-09-19T05:41:11.198681+0000","last_became_active":"2024-09-19T05:39:08.948641+0000","last_became_peered":"2024-09-19T05:39:08.948641+0000","last_unstale":"2024-09-19T05:41:11.198681+0000","last_undegraded":"2024-09-19T05:41:11.198681+0000","last_fullsized":"2024-09-19T05:41:11.198681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:14.845938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198437+0000","last_change":"2024-09-19T05:39:01.893259+0000","last_active":"2024-09-19T05:41:11.198437+0000","last_peered":"2024-09-19T05:41:11.198437+0000","last_clean":"2024-09-19T05:41:11.198437+0000","last_became_active":"2024-09-19T05:39:01.892974+0000","last_became_peered":"2024-09-19T05:39:01.892974+0000","last_unstale":"2024-09-19T05:41:11.198437+0000","last_undegraded":"2024-09-19T05:41:11.198437+0000","last_fullsized":"2024-09-19T05:41:11.198437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:46.493638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208655+0000","last_change":"2024-09-19T05:38:59.855832+0000","last_active":"2024-09-19T05:41:11.208655+0000","last_peered":"2024-09-19T05:41:11.208655+0000","last_clean":"2024-09-19T05:41:11.208655+0000","last_became_active":"2024-09-19T05:38:59.855500+0000","last_became_peered":"2024-09-19T05:38:59.855500+0000","last_unstale":"2024-09-19T05:41:11.208655+0000","last_undegraded":"2024-09-19T05:41:11.208655+0000","last_fullsized":"2024-09-19T05:41:11.208655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:44.598206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197628+0000","last_change":"2024-09-19T05:39:08.949755+0000","last_active":"2024-09-19T05:41:11.197628+0000","last_peered":"2024-09-19T05:41:11.197628+0000","last_clean":"2024-09-19T05:41:11.197628+0000","last_became_active":"2024-09-19T05:39:08.949332+0000","last_became_peered":"2024-09-19T05:39:08.949332+0000","last_unstale":"2024-09-19T05:41:11.197628+0000","last_undegraded":"2024-09-19T05:41:11.197628+0000","last_fullsized":"2024-09-19T05:41:11.197628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208477+0000","last_change":"2024-09-19T05:39:01.899924+0000","last_active":"2024-09-19T05:41:11.208477+0000","last_peered":"2024-09-19T05:41:11.208477+0000","last_clean":"2024-09-19T05:41:11.208477+0000","last_became_active":"2024-09-19T05:39:01.899591+0000","last_became_peered":"2024-09-19T05:39:01.899591+0000","last_unstale":"2024-09-19T05:41:11.208477+0000","last_undegraded":"2024-09-19T05:41:11.208477+0000","last_fullsized":"2024-09-19T05:41:11.208477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:36.843470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208971+0000","last_change":"2024-09-19T05:38:59.854094+0000","last_active":"2024-09-19T05:41:11.208971+0000","last_peered":"2024-09-19T05:41:11.208971+0000","last_clean":"2024-09-19T05:41:11.208971+0000","last_became_active":"2024-09-19T05:38:59.853788+0000","last_became_peered":"2024-09-19T05:38:59.853788+0000","last_unstale":"2024-09-19T05:41:11.208971+0000","last_undegraded":"2024-09-19T05:41:11.208971+0000","last_fullsized":"2024-09-19T05:41:11.208971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:51.343069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209693+0000","last_change":"2024-09-19T05:38:59.854756+0000","last_active":"2024-09-19T05:41:11.209693+0000","last_peered":"2024-09-19T05:41:11.209693+0000","last_clean":"2024-09-19T05:41:11.209693+0000","last_became_active":"2024-09-19T05:38:59.854405+0000","last_became_peered":"2024-09-19T05:38:59.854405+0000","last_unstale":"2024-09-19T05:41:11.209693+0000","last_undegraded":"2024-09-19T05:41:11.209693+0000","last_fullsized":"2024-09-19T05:41:11.209693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:05.385105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208811+0000","last_change":"2024-09-19T05:39:01.887206+0000","last_active":"2024-09-19T05:41:11.208811+0000","last_peered":"2024-09-19T05:41:11.208811+0000","last_clean":"2024-09-19T05:41:11.208811+0000","last_became_active":"2024-09-19T05:39:01.886544+0000","last_became_peered":"2024-09-19T05:39:01.886544+0000","last_unstale":"2024-09-19T05:41:11.208811+0000","last_undegraded":"2024-09-19T05:41:11.208811+0000","last_fullsized":"2024-09-19T05:41:11.208811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:19.430988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195629+0000","last_change":"2024-09-19T05:40:10.596620+0000","last_active":"2024-09-19T05:41:11.195629+0000","last_peered":"2024-09-19T05:41:11.195629+0000","last_clean":"2024-09-19T05:41:11.195629+0000","last_became_active":"2024-09-19T05:40:10.596126+0000","last_became_peered":"2024-09-19T05:40:10.596126+0000","last_unstale":"2024-09-19T05:41:11.195629+0000","last_undegraded":"2024-09-19T05:41:11.195629+0000","last_fullsized":"2024-09-19T05:41:11.195629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:33:16.573196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:32.260880+0000","last_change":"2024-09-19T05:38:59.853714+0000","last_active":"2024-09-19T05:41:32.260880+0000","last_peered":"2024-09-19T05:41:32.260880+0000","last_clean":"2024-09-19T05:41:32.260880+0000","last_became_active":"2024-09-19T05:38:59.853330+0000","last_became_peered":"2024-09-19T05:38:59.853330+0000","last_unstale":"2024-09-19T05:41:32.260880+0000","last_undegraded":"2024-09-19T05:41:32.260880+0000","last_fullsized":"2024-09-19T05:41:32.260880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:38:58.838016+0000","last_clean_scrub_stamp":"2024-09-19T05:38:58.838016+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:38:54.858381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199935+0000","last_change":"2024-09-19T05:40:10.590527+0000","last_active":"2024-09-19T05:41:11.199935+0000","last_peered":"2024-09-19T05:41:11.199935+0000","last_clean":"2024-09-19T05:41:11.199935+0000","last_became_active":"2024-09-19T05:40:10.590181+0000","last_became_peered":"2024-09-19T05:40:10.590181+0000","last_unstale":"2024-09-19T05:41:11.199935+0000","last_undegraded":"2024-09-19T05:41:11.199935+0000","last_fullsized":"2024-09-19T05:41:11.199935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:22:55.157416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195758+0000","last_change":"2024-09-19T05:39:08.950056+0000","last_active":"2024-09-19T05:41:11.195758+0000","last_peered":"2024-09-19T05:41:11.195758+0000","last_clean":"2024-09-19T05:41:11.195758+0000","last_became_active":"2024-09-19T05:39:08.949454+0000","last_became_peered":"2024-09-19T05:39:08.949454+0000","last_unstale":"2024-09-19T05:41:11.195758+0000","last_undegraded":"2024-09-19T05:41:11.195758+0000","last_fullsized":"2024-09-19T05:41:11.195758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:07.928775+0000","last_clean_scrub_stamp":"2024-09-19T05:39:07.928775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:30.500747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197193+0000","last_change":"2024-09-19T05:39:01.903341+0000","last_active":"2024-09-19T05:41:11.197193+0000","last_peered":"2024-09-19T05:41:11.197193+0000","last_clean":"2024-09-19T05:41:11.197193+0000","last_became_active":"2024-09-19T05:39:01.902900+0000","last_became_peered":"2024-09-19T05:39:01.902900+0000","last_unstale":"2024-09-19T05:41:11.197193+0000","last_undegraded":"2024-09-19T05:41:11.197193+0000","last_fullsized":"2024-09-19T05:41:11.197193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:43:50.332133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195772+0000","last_change":"2024-09-19T05:39:01.900069+0000","last_active":"2024-09-19T05:41:11.195772+0000","last_peered":"2024-09-19T05:41:11.195772+0000","last_clean":"2024-09-19T05:41:11.195772+0000","last_became_active":"2024-09-19T05:39:01.899745+0000","last_became_peered":"2024-09-19T05:39:01.899745+0000","last_unstale":"2024-09-19T05:41:11.195772+0000","last_undegraded":"2024-09-19T05:41:11.195772+0000","last_fullsized":"2024-09-19T05:41:11.195772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:07.236350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195298+0000","last_change":"2024-09-19T05:40:10.893573+0000","last_active":"2024-09-19T05:41:11.195298+0000","last_peered":"2024-09-19T05:41:11.195298+0000","last_clean":"2024-09-19T05:41:11.195298+0000","last_became_active":"2024-09-19T05:40:10.893216+0000","last_became_peered":"2024-09-19T05:40:10.893216+0000","last_unstale":"2024-09-19T05:41:11.195298+0000","last_undegraded":"2024-09-19T05:41:11.195298+0000","last_fullsized":"2024-09-19T05:41:11.195298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:50.147170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198667+0000","last_change":"2024-09-19T05:39:01.895833+0000","last_active":"2024-09-19T05:41:11.198667+0000","last_peered":"2024-09-19T05:41:11.198667+0000","last_clean":"2024-09-19T05:41:11.198667+0000","last_became_active":"2024-09-19T05:39:01.895532+0000","last_became_peered":"2024-09-19T05:39:01.895532+0000","last_unstale":"2024-09-19T05:41:11.198667+0000","last_undegraded":"2024-09-19T05:41:11.198667+0000","last_fullsized":"2024-09-19T05:41:11.198667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:16.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209643+0000","last_change":"2024-09-19T05:39:01.891039+0000","last_active":"2024-09-19T05:41:11.209643+0000","last_peered":"2024-09-19T05:41:11.209643+0000","last_clean":"2024-09-19T05:41:11.209643+0000","last_became_active":"2024-09-19T05:39:01.890657+0000","last_became_peered":"2024-09-19T05:39:01.890657+0000","last_unstale":"2024-09-19T05:41:11.209643+0000","last_undegraded":"2024-09-19T05:41:11.209643+0000","last_fullsized":"2024-09-19T05:41:11.209643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:50.242353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198638+0000","last_change":"2024-09-19T05:39:01.899057+0000","last_active":"2024-09-19T05:41:11.198638+0000","last_peered":"2024-09-19T05:41:11.198638+0000","last_clean":"2024-09-19T05:41:11.198638+0000","last_became_active":"2024-09-19T05:39:01.898537+0000","last_became_peered":"2024-09-19T05:39:01.898537+0000","last_unstale":"2024-09-19T05:41:11.198638+0000","last_undegraded":"2024-09-19T05:41:11.198638+0000","last_fullsized":"2024-09-19T05:41:11.198638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:32.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195938+0000","last_change":"2024-09-19T05:39:01.903970+0000","last_active":"2024-09-19T05:41:11.195938+0000","last_peered":"2024-09-19T05:41:11.195938+0000","last_clean":"2024-09-19T05:41:11.195938+0000","last_became_active":"2024-09-19T05:39:01.902656+0000","last_became_peered":"2024-09-19T05:39:01.902656+0000","last_unstale":"2024-09-19T05:41:11.195938+0000","last_undegraded":"2024-09-19T05:41:11.195938+0000","last_fullsized":"2024-09-19T05:41:11.195938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:57.992383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208765+0000","last_change":"2024-09-19T05:39:01.891286+0000","last_active":"2024-09-19T05:41:11.208765+0000","last_peered":"2024-09-19T05:41:11.208765+0000","last_clean":"2024-09-19T05:41:11.208765+0000","last_became_active":"2024-09-19T05:39:01.890979+0000","last_became_peered":"2024-09-19T05:39:01.890979+0000","last_unstale":"2024-09-19T05:41:11.208765+0000","last_undegraded":"2024-09-19T05:41:11.208765+0000","last_fullsized":"2024-09-19T05:41:11.208765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:46.726814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208449+0000","last_change":"2024-09-19T05:39:01.890597+0000","last_active":"2024-09-19T05:41:11.208449+0000","last_peered":"2024-09-19T05:41:11.208449+0000","last_clean":"2024-09-19T05:41:11.208449+0000","last_became_active":"2024-09-19T05:39:01.887508+0000","last_became_peered":"2024-09-19T05:39:01.887508+0000","last_unstale":"2024-09-19T05:41:11.208449+0000","last_undegraded":"2024-09-19T05:41:11.208449+0000","last_fullsized":"2024-09-19T05:41:11.208449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:28.508091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208803+0000","last_change":"2024-09-19T05:39:01.900059+0000","last_active":"2024-09-19T05:41:11.208803+0000","last_peered":"2024-09-19T05:41:11.208803+0000","last_clean":"2024-09-19T05:41:11.208803+0000","last_became_active":"2024-09-19T05:39:01.897955+0000","last_became_peered":"2024-09-19T05:39:01.897955+0000","last_unstale":"2024-09-19T05:41:11.208803+0000","last_undegraded":"2024-09-19T05:41:11.208803+0000","last_fullsized":"2024-09-19T05:41:11.208803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:42.524567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210072+0000","last_change":"2024-09-19T05:39:01.896883+0000","last_active":"2024-09-19T05:41:11.210072+0000","last_peered":"2024-09-19T05:41:11.210072+0000","last_clean":"2024-09-19T05:41:11.210072+0000","last_became_active":"2024-09-19T05:39:01.896574+0000","last_became_peered":"2024-09-19T05:39:01.896574+0000","last_unstale":"2024-09-19T05:41:11.210072+0000","last_undegraded":"2024-09-19T05:41:11.210072+0000","last_fullsized":"2024-09-19T05:41:11.210072+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:26.833041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209689+0000","last_change":"2024-09-19T05:39:01.897131+0000","last_active":"2024-09-19T05:41:11.209689+0000","last_peered":"2024-09-19T05:41:11.209689+0000","last_clean":"2024-09-19T05:41:11.209689+0000","last_became_active":"2024-09-19T05:39:01.896831+0000","last_became_peered":"2024-09-19T05:39:01.896831+0000","last_unstale":"2024-09-19T05:41:11.209689+0000","last_undegraded":"2024-09-19T05:41:11.209689+0000","last_fullsized":"2024-09-19T05:41:11.209689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:38.260308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197591+0000","last_change":"2024-09-19T05:39:01.901525+0000","last_active":"2024-09-19T05:41:11.197591+0000","last_peered":"2024-09-19T05:41:11.197591+0000","last_clean":"2024-09-19T05:41:11.197591+0000","last_became_active":"2024-09-19T05:39:01.901127+0000","last_became_peered":"2024-09-19T05:39:01.901127+0000","last_unstale":"2024-09-19T05:41:11.197591+0000","last_undegraded":"2024-09-19T05:41:11.197591+0000","last_fullsized":"2024-09-19T05:41:11.197591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:31.861218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.196030+0000","last_change":"2024-09-19T05:39:01.891211+0000","last_active":"2024-09-19T05:41:11.196030+0000","last_peered":"2024-09-19T05:41:11.196030+0000","last_clean":"2024-09-19T05:41:11.196030+0000","last_became_active":"2024-09-19T05:39:01.890149+0000","last_became_peered":"2024-09-19T05:39:01.890149+0000","last_unstale":"2024-09-19T05:41:11.196030+0000","last_undegraded":"2024-09-19T05:41:11.196030+0000","last_fullsized":"2024-09-19T05:41:11.196030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:15.897239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199711+0000","last_change":"2024-09-19T05:39:01.897912+0000","last_active":"2024-09-19T05:41:11.199711+0000","last_peered":"2024-09-19T05:41:11.199711+0000","last_clean":"2024-09-19T05:41:11.199711+0000","last_became_active":"2024-09-19T05:39:01.897532+0000","last_became_peered":"2024-09-19T05:39:01.897532+0000","last_unstale":"2024-09-19T05:41:11.199711+0000","last_undegraded":"2024-09-19T05:41:11.199711+0000","last_fullsized":"2024-09-19T05:41:11.199711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:22.907434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197870+0000","last_change":"2024-09-19T05:39:01.906376+0000","last_active":"2024-09-19T05:41:11.197870+0000","last_peered":"2024-09-19T05:41:11.197870+0000","last_clean":"2024-09-19T05:41:11.197870+0000","last_became_active":"2024-09-19T05:39:01.906077+0000","last_became_peered":"2024-09-19T05:39:01.906077+0000","last_unstale":"2024-09-19T05:41:11.197870+0000","last_undegraded":"2024-09-19T05:41:11.197870+0000","last_fullsized":"2024-09-19T05:41:11.197870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:27.590311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209348+0000","last_change":"2024-09-19T05:39:01.895752+0000","last_active":"2024-09-19T05:41:11.209348+0000","last_peered":"2024-09-19T05:41:11.209348+0000","last_clean":"2024-09-19T05:41:11.209348+0000","last_became_active":"2024-09-19T05:39:01.895425+0000","last_became_peered":"2024-09-19T05:39:01.895425+0000","last_unstale":"2024-09-19T05:41:11.209348+0000","last_undegraded":"2024-09-19T05:41:11.209348+0000","last_fullsized":"2024-09-19T05:41:11.209348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:28:24.283867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198183+0000","last_change":"2024-09-19T05:39:01.893029+0000","last_active":"2024-09-19T05:41:11.198183+0000","last_peered":"2024-09-19T05:41:11.198183+0000","last_clean":"2024-09-19T05:41:11.198183+0000","last_became_active":"2024-09-19T05:39:01.892732+0000","last_became_peered":"2024-09-19T05:39:01.892732+0000","last_unstale":"2024-09-19T05:41:11.198183+0000","last_undegraded":"2024-09-19T05:41:11.198183+0000","last_fullsized":"2024-09-19T05:41:11.198183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:09.526292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198273+0000","last_change":"2024-09-19T05:39:01.894750+0000","last_active":"2024-09-19T05:41:11.198273+0000","last_peered":"2024-09-19T05:41:11.198273+0000","last_clean":"2024-09-19T05:41:11.198273+0000","last_became_active":"2024-09-19T05:39:01.894432+0000","last_became_peered":"2024-09-19T05:39:01.894432+0000","last_unstale":"2024-09-19T05:41:11.198273+0000","last_undegraded":"2024-09-19T05:41:11.198273+0000","last_fullsized":"2024-09-19T05:41:11.198273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:57.479614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209399+0000","last_change":"2024-09-19T05:39:01.894474+0000","last_active":"2024-09-19T05:41:11.209399+0000","last_peered":"2024-09-19T05:41:11.209399+0000","last_clean":"2024-09-19T05:41:11.209399+0000","last_became_active":"2024-09-19T05:39:01.894150+0000","last_became_peered":"2024-09-19T05:39:01.894150+0000","last_unstale":"2024-09-19T05:41:11.209399+0000","last_undegraded":"2024-09-19T05:41:11.209399+0000","last_fullsized":"2024-09-19T05:41:11.209399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:32.172539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199326+0000","last_change":"2024-09-19T05:40:10.590802+0000","last_active":"2024-09-19T05:41:11.199326+0000","last_peered":"2024-09-19T05:41:11.199326+0000","last_clean":"2024-09-19T05:41:11.199326+0000","last_became_active":"2024-09-19T05:40:10.590515+0000","last_became_peered":"2024-09-19T05:40:10.590515+0000","last_unstale":"2024-09-19T05:41:11.199326+0000","last_undegraded":"2024-09-19T05:41:11.199326+0000","last_fullsized":"2024-09-19T05:41:11.199326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:37.010052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.200430+0000","last_change":"2024-09-19T05:39:01.898144+0000","last_active":"2024-09-19T05:41:11.200430+0000","last_peered":"2024-09-19T05:41:11.200430+0000","last_clean":"2024-09-19T05:41:11.200430+0000","last_became_active":"2024-09-19T05:39:01.897794+0000","last_became_peered":"2024-09-19T05:39:01.897794+0000","last_unstale":"2024-09-19T05:41:11.200430+0000","last_undegraded":"2024-09-19T05:41:11.200430+0000","last_fullsized":"2024-09-19T05:41:11.200430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:10.953097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209676+0000","last_change":"2024-09-19T05:39:01.900628+0000","last_active":"2024-09-19T05:41:11.209676+0000","last_peered":"2024-09-19T05:41:11.209676+0000","last_clean":"2024-09-19T05:41:11.209676+0000","last_became_active":"2024-09-19T05:39:01.900251+0000","last_became_peered":"2024-09-19T05:39:01.900251+0000","last_unstale":"2024-09-19T05:41:11.209676+0000","last_undegraded":"2024-09-19T05:41:11.209676+0000","last_fullsized":"2024-09-19T05:41:11.209676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:05.450748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.210113+0000","last_change":"2024-09-19T05:39:01.901990+0000","last_active":"2024-09-19T05:41:11.210113+0000","last_peered":"2024-09-19T05:41:11.210113+0000","last_clean":"2024-09-19T05:41:11.210113+0000","last_became_active":"2024-09-19T05:39:01.901656+0000","last_became_peered":"2024-09-19T05:39:01.901656+0000","last_unstale":"2024-09-19T05:41:11.210113+0000","last_undegraded":"2024-09-19T05:41:11.210113+0000","last_fullsized":"2024-09-19T05:41:11.210113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:42.910677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.208426+0000","last_change":"2024-09-19T05:39:01.896623+0000","last_active":"2024-09-19T05:41:11.208426+0000","last_peered":"2024-09-19T05:41:11.208426+0000","last_clean":"2024-09-19T05:41:11.208426+0000","last_became_active":"2024-09-19T05:39:01.896328+0000","last_became_peered":"2024-09-19T05:39:01.896328+0000","last_unstale":"2024-09-19T05:41:11.208426+0000","last_undegraded":"2024-09-19T05:41:11.208426+0000","last_fullsized":"2024-09-19T05:41:11.208426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:46.667558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199713+0000","last_change":"2024-09-19T05:39:01.885743+0000","last_active":"2024-09-19T05:41:11.199713+0000","last_peered":"2024-09-19T05:41:11.199713+0000","last_clean":"2024-09-19T05:41:11.199713+0000","last_became_active":"2024-09-19T05:39:01.885324+0000","last_became_peered":"2024-09-19T05:39:01.885324+0000","last_unstale":"2024-09-19T05:41:11.199713+0000","last_undegraded":"2024-09-19T05:41:11.199713+0000","last_fullsized":"2024-09-19T05:41:11.199713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:52.042975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.209595+0000","last_change":"2024-09-19T05:39:01.900277+0000","last_active":"2024-09-19T05:41:11.209595+0000","last_peered":"2024-09-19T05:41:11.209595+0000","last_clean":"2024-09-19T05:41:11.209595+0000","last_became_active":"2024-09-19T05:39:01.899844+0000","last_became_peered":"2024-09-19T05:39:01.899844+0000","last_unstale":"2024-09-19T05:41:11.209595+0000","last_undegraded":"2024-09-19T05:41:11.209595+0000","last_fullsized":"2024-09-19T05:41:11.209595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:11.184498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.199916+0000","last_change":"2024-09-19T05:39:01.895419+0000","last_active":"2024-09-19T05:41:11.199916+0000","last_peered":"2024-09-19T05:41:11.199916+0000","last_clean":"2024-09-19T05:41:11.199916+0000","last_became_active":"2024-09-19T05:39:01.895069+0000","last_became_peered":"2024-09-19T05:39:01.895069+0000","last_unstale":"2024-09-19T05:41:11.199916+0000","last_undegraded":"2024-09-19T05:41:11.199916+0000","last_fullsized":"2024-09-19T05:41:11.199916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:32.531454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.195219+0000","last_change":"2024-09-19T05:39:01.898249+0000","last_active":"2024-09-19T05:41:11.195219+0000","last_peered":"2024-09-19T05:41:11.195219+0000","last_clean":"2024-09-19T05:41:11.195219+0000","last_became_active":"2024-09-19T05:39:01.897911+0000","last_became_peered":"2024-09-19T05:39:01.897911+0000","last_unstale":"2024-09-19T05:41:11.195219+0000","last_undegraded":"2024-09-19T05:41:11.195219+0000","last_fullsized":"2024-09-19T05:41:11.195219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:24.079303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198352+0000","last_change":"2024-09-19T05:39:01.895520+0000","last_active":"2024-09-19T05:41:11.198352+0000","last_peered":"2024-09-19T05:41:11.198352+0000","last_clean":"2024-09-19T05:41:11.198352+0000","last_became_active":"2024-09-19T05:39:01.895165+0000","last_became_peered":"2024-09-19T05:39:01.895165+0000","last_unstale":"2024-09-19T05:41:11.198352+0000","last_undegraded":"2024-09-19T05:41:11.198352+0000","last_fullsized":"2024-09-19T05:41:11.198352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:44.268853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.198813+0000","last_change":"2024-09-19T05:39:01.899401+0000","last_active":"2024-09-19T05:41:11.198813+0000","last_peered":"2024-09-19T05:41:11.198813+0000","last_clean":"2024-09-19T05:41:11.198813+0000","last_became_active":"2024-09-19T05:39:01.899054+0000","last_became_peered":"2024-09-19T05:39:01.899054+0000","last_unstale":"2024-09-19T05:41:11.198813+0000","last_undegraded":"2024-09-19T05:41:11.198813+0000","last_fullsized":"2024-09-19T05:41:11.198813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:21.179914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T05:41:11.197743+0000","last_change":"2024-09-19T05:39:01.904483+0000","last_active":"2024-09-19T05:41:11.197743+0000","last_peered":"2024-09-19T05:41:11.197743+0000","last_clean":"2024-09-19T05:41:11.197743+0000","last_became_active":"2024-09-19T05:39:01.904148+0000","last_became_peered":"2024-09-19T05:39:01.904148+0000","last_unstale":"2024-09-19T05:41:11.197743+0000","last_undegraded":"2024-09-19T05:41:11.197743+0000","last_fullsized":"2024-09-19T05:41:11.197743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:39:00.870448+0000","last_clean_scrub_stamp":"2024-09-19T05:39:00.870448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:15.973564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74045,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1095,"num_read_kb":628,"num_write":273,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1816,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":230763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547242,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":519275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1170850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":522347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168635,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":518517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1667072,"data_stored":1116337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1048576,"data_stored":514289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1675264,"data_stored":1110851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":520375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":513463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":579548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":580498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":516053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:41:36.484 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:41:36.741 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 -- 172.21.15.38:0/1626348346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0101420 msgr2=0x7f02c010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 --2- 172.21.15.38:0/1626348346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0101420 0x7f02c010d950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009cd0 tx=0x7f02b802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 -- 172.21.15.38:0/1626348346 shutdown_connections 2024-09-19T05:41:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 --2- 172.21.15.38:0/1626348346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0101420 0x7f02c010d950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 --2- 172.21.15.38:0/1626348346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 0x7f02c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 --2- 172.21.15.38:0/1626348346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0106a80 0x7f02c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 -- 172.21.15.38:0/1626348346 >> 172.21.15.38:0/1626348346 conn(0x7f02c00fc7a0 msgr2=0x7f02c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 -- 172.21.15.38:0/1626348346 shutdown_connections 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 -- 172.21.15.38:0/1626348346 wait complete. 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.207+0000 7f02c8109640 1 Processor -- start 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 -- start start 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 0x7f02c0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0101420 0x7f02c0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02c0110e80 con 0x7f02c0106a80 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f02c0110d00 con 0x7f02c0100a80 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c8109640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f02c0111000 con 0x7f02c0101420 2024-09-19T05:41:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59810/0 (socket says 172.21.15.38:59810) 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 -- 172.21.15.38:0/1533031492 learned_addr learned my addr 172.21.15.38:0/1533031492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c5ea5640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 0x7f02c0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c56a4640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0101420 0x7f02c0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 -- 172.21.15.38:0/1533031492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0101420 msgr2=0x7f02c0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0101420 0x7f02c0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 -- 172.21.15.38:0/1533031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 msgr2=0x7f02c0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 0x7f02c0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 -- 172.21.15.38:0/1533031492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c019d3e0 con 0x7f02c0106a80 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.208+0000 7f02c66a6640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005cc0 tx=0x7f02b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.209+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8004040 con 0x7f02c0106a80 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.209+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b80046c0 con 0x7f02c0106a80 2024-09-19T05:41:37.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.209+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c019d670 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.209+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8046480 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.209+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c01a4fb0 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.210+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c0100ee0 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.211+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f02b80305a0 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.212+0000 7f02b6ffd640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 0x7f02ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.212+0000 7f02c5ea5640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 0x7f02ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.212+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f02b802e030 con 0x7f02c0106a80 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.212+0000 7f02c5ea5640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 0x7f02ac0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0058c0 tx=0x7f02bc00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:37.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.215+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02b804b060 con 0x7f02c0106a80 2024-09-19T05:41:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:37 smithi038 ceph-mon[31227]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:41:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:37 smithi038 ceph-mon[31227]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.428+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c0199790 con 0x7f02c0106a80 2024-09-19T05:41:37.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.429+0000 7f02b6ffd640 1 -- 172.21.15.38:0/1533031492 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f02c0199790 con 0x7f02c0106a80 2024-09-19T05:41:37.431 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:37.431 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:41:37.431 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 msgr2=0x7f02ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 0x7f02ac0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0058c0 tx=0x7f02bc00a250 comp rx=0 tx=0).stop 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 msgr2=0x7f02c019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005cc0 tx=0x7f02b8003970 comp rx=0 tx=0).stop 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 shutdown_connections 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f02ac0a32c0 0x7f02ac0a5780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02c0106a80 0x7f02c019cd00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c0101420 0x7f02c0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 --2- 172.21.15.38:0/1533031492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02c0100a80 0x7f02c0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 >> 172.21.15.38:0/1533031492 conn(0x7f02c00fc7a0 msgr2=0x7f02c006a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 shutdown_connections 2024-09-19T05:41:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:37.431+0000 7f02c8109640 1 -- 172.21.15.38:0/1533031492 wait complete. 2024-09-19T05:41:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:37 smithi179 ceph-mon[35781]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:41:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:37 smithi179 ceph-mon[35781]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:37 smithi047 ceph-mon[35327]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:41:37.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:37 smithi047 ceph-mon[35327]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:41:38.166 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:41:38.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:38 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1533031492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:38.429 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:38.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:38 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1533031492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:38.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:38 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1533031492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.959+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2172156607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 msgr2=0x7f40e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.959+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2172156607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e010ea90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009ca0 tx=0x7f40d802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2172156607 shutdown_connections 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2172156607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e010ea90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2172156607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 0x7f40e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2172156607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 0x7f40e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2172156607 >> 172.21.15.38:0/2172156607 conn(0x7f40e00fd8c0 msgr2=0x7f40e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2172156607 shutdown_connections 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2172156607 wait complete. 2024-09-19T05:41:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 Processor -- start 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.960+0000 7f40e4ae4640 1 -- start start 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 0x7f40e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 0x7f40e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f40e0111f70 con 0x7f40e0106f10 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40e0111df0 con 0x7f40e0101de0 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40e4ae4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40e01120f0 con 0x7f40e0102790 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35130/0 (socket says 172.21.15.38:35130) 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 -- 172.21.15.38:0/2747146200 learned_addr learned my addr 172.21.15.38:0/2747146200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40d7fff640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 0x7f40e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:38.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40de59c640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 0x7f40e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 -- 172.21.15.38:0/2747146200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 msgr2=0x7f40e0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 0x7f40e0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 -- 172.21.15.38:0/2747146200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 msgr2=0x7f40e0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 0x7f40e0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 -- 172.21.15.38:0/2747146200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e019d2e0 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.961+0000 7f40ded9d640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40d8031a20 tx=0x7f40d8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.962+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8003b70 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.962+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e019d570 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.962+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d8003d10 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.962+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d802fc80 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.962+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e01a4eb0 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.963+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e01021c0 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.964+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f40d80046a0 con 0x7f40e0106f10 2024-09-19T05:41:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.964+0000 7f40d77fe640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 0x7f40cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.965+0000 7f40de59c640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 0x7f40cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.965+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f40d80f0e80 con 0x7f40e0106f10 2024-09-19T05:41:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.965+0000 7f40de59c640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 0x7f40cc0a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40d000abe0 tx=0x7f40d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:38.969+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f40d80bd870 con 0x7f40e0106f10 2024-09-19T05:41:39.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.180+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40e010a150 con 0x7f40e0106f10 2024-09-19T05:41:39.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.181+0000 7f40d77fe640 1 -- 172.21.15.38:0/2747146200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f40d80bd170 con 0x7f40e0106f10 2024-09-19T05:41:39.183 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:39.183 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:41:39.183 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 msgr2=0x7f40cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 0x7f40cc0a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40d000abe0 tx=0x7f40d000a2d0 comp rx=0 tx=0).stop 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 msgr2=0x7f40e019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40d8031a20 tx=0x7f40d8031a50 comp rx=0 tx=0).stop 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 shutdown_connections 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f40cc0a3240 0x7f40cc0a5700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40e0106f10 0x7f40e019cc00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:39.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40e0102790 0x7f40e0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:39.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 --2- 172.21.15.38:0/2747146200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40e0101de0 0x7f40e0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:39.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 >> 172.21.15.38:0/2747146200 conn(0x7f40e00fd8c0 msgr2=0x7f40e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:39.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 shutdown_connections 2024-09-19T05:41:39.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:39.183+0000 7f40e4ae4640 1 -- 172.21.15.38:0/2747146200 wait complete. 2024-09-19T05:41:39.427 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:39 smithi038 ceph-mon[31227]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:41:39.427 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:39 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2747146200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:39 smithi179 ceph-mon[35781]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:41:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:39 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2747146200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:39.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:39 smithi047 ceph-mon[35327]: pgmap v365: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:41:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:39 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2747146200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:39.905 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:41:40.170 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 -- 172.21.15.38:0/2181492264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102ef0 msgr2=0x7fa0a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 --2- 172.21.15.38:0/2181492264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102ef0 0x7fa0a010da40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa098009ca0 tx=0x7fa09802a950 comp rx=0 tx=0).stop 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 -- 172.21.15.38:0/2181492264 shutdown_connections 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 --2- 172.21.15.38:0/2181492264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102ef0 0x7fa0a010da40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 --2- 172.21.15.38:0/2181492264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0102570 0x7fa0a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 --2- 172.21.15.38:0/2181492264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0100e20 0x7fa0a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.680+0000 7fa0a763f640 1 -- 172.21.15.38:0/2181492264 >> 172.21.15.38:0/2181492264 conn(0x7fa0a00fc7d0 msgr2=0x7fa0a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 -- 172.21.15.38:0/2181492264 shutdown_connections 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 -- 172.21.15.38:0/2181492264 wait complete. 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 Processor -- start 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 -- start start 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0100e20 0x7fa0a01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.681+0000 7fa0a763f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 0x7fa0a019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a763f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0110ff0 con 0x7fa0a0102570 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a763f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0110e70 con 0x7fa0a0100e20 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a763f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0111170 con 0x7fa0a0102ef0 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35144/0 (socket says 172.21.15.38:35144) 2024-09-19T05:41:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 -- 172.21.15.38:0/1538575902 learned_addr learned my addr 172.21.15.38:0/1538575902 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a53db640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0100e20 0x7fa0a01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a5bdc640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 0x7fa0a019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 -- 172.21.15.38:0/1538575902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 msgr2=0x7fa0a019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 0x7fa0a019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 -- 172.21.15.38:0/1538575902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0100e20 msgr2=0x7fa0a01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0100e20 0x7fa0a01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 -- 172.21.15.38:0/1538575902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a019d490 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a5bdc640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 0x7fa0a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a4bda640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d950 tx=0x7fa09000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090017030 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a019d780 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa090004020 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.682+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090005650 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.683+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a01a5060 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.683+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a0101200 con 0x7fa0a0102570 2024-09-19T05:41:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.686+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa0900041d0 con 0x7fa0a0102570 2024-09-19T05:41:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.686+0000 7fa0967fc640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 0x7fa08c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.686+0000 7fa0a53db640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 0x7fa08c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.686+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fa0900cfe40 con 0x7fa0a0102570 2024-09-19T05:41:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.687+0000 7fa0a53db640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 0x7fa08c0a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009990 tx=0x7fa09c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.690+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa090024330 con 0x7fa0a0102570 2024-09-19T05:41:40.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.867+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0a0199840 con 0x7fa0a0102570 2024-09-19T05:41:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.869+0000 7fa0967fc640 1 -- 172.21.15.38:0/1538575902 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24875 (secure 0 0 0) 0x7fa0a0199840 con 0x7fa0a0102570 2024-09-19T05:41:40.871 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:40.872 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:41:11.190669+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:41:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.871+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 msgr2=0x7fa08c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.871+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 0x7fa08c0a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009990 tx=0x7fa09c009110 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 msgr2=0x7fa0a0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d950 tx=0x7fa09000dd50 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 shutdown_connections 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa08c0a3240 0x7fa08c0a5700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a0102ef0 0x7fa0a019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0a0102570 0x7fa0a0198a20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 --2- 172.21.15.38:0/1538575902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a0100e20 0x7fa0a01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 >> 172.21.15.38:0/1538575902 conn(0x7fa0a00fc7d0 msgr2=0x7fa0a01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 shutdown_connections 2024-09-19T05:41:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:40.872+0000 7fa0a763f640 1 -- 172.21.15.38:0/1538575902 wait complete. 2024-09-19T05:41:41.578 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:41 smithi038 ceph-mon[31227]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:41:41.578 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:41 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1538575902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:41.607 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T05:41:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:41 smithi179 ceph-mon[35781]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:41:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:41 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1538575902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:41.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:41 smithi047 ceph-mon[35327]: pgmap v366: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:41:41.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:41 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1538575902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:41:41.866 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:43.096 INFO:teuthology.orchestra.run.smithi038.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T05:41:43.096 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T05:41:43.096 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T05:41:43.097 DEBUG:tasks.quiescer:Yielding 2024-09-19T05:41:43.097 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T05:41:43.110 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:41:43.111 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T05:41:43.111 INFO:tasks.workunit:timeout=3h 2024-09-19T05:41:43.111 INFO:tasks.workunit:cleanup=True 2024-09-19T05:41:43.111 DEBUG:teuthology.orchestra.run.smithi038:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:43.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:41:43.131 INFO:teuthology.orchestra.run.smithi038.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:43.131 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T05:41:43.131 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T05:41:43.131 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:43.132 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T05:41:43.132 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-19 05:39:54.800825150 +0000 2024-09-19T05:41:43.132 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-19 05:39:54.800825150 +0000 2024-09-19T05:41:43.132 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-19 05:41:29.803650604 +0000 2024-09-19T05:41:43.132 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-19 05:39:54.800825150 +0000 2024-09-19T05:41:43.132 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T05:41:43.132 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T05:41:43.211 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:41:43.269 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T05:41:43.382 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:43.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:43 smithi038 ceph-mon[31227]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:41:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:43 smithi179 ceph-mon[35781]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:41:43.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:43 smithi047 ceph-mon[35327]: pgmap v367: 129 pgs: 129 active+clean; 653 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:41:43.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/1160417052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 msgr2=0x7f2f8810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/1160417052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f8810ec50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009cd0 tx=0x7f2f8002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/1160417052 shutdown_connections 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/1160417052 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f8810ec50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/1160417052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 0x7f2f88107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/1160417052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 0x7f2f88103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/1160417052 >> 172.21.15.38:0/1160417052 conn(0x7f2f880fc7d0 msgr2=0x7f2f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/1160417052 shutdown_connections 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.859+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/1160417052 wait complete. 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.860+0000 7f2f8c8c7640 1 Processor -- start 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.860+0000 7f2f8c8c7640 1 -- start start 2024-09-19T05:41:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 0x7f2f88100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 0x7f2f88100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f881120b0 con 0x7f2f881082e0 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88111f30 con 0x7f2f88103760 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8c8c7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88112230 con 0x7f2f88102db0 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35158/0 (socket says 172.21.15.38:35158) 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f85d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 0x7f2f88100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 -- 172.21.15.38:0/4178702502 learned_addr learned my addr 172.21.15.38:0/4178702502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f8659c640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 0x7f2f88100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 -- 172.21.15.38:0/4178702502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 msgr2=0x7f2f88100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 0x7f2f88100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 -- 172.21.15.38:0/4178702502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 msgr2=0x7f2f88100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 0x7f2f88100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 -- 172.21.15.38:0/4178702502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881a6a70 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.861+0000 7f2f86d9d640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005cc0 tx=0x7f2f800093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.862+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80004040 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.862+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a6d00 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.862+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f800046c0 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.862+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a7290 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.862+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80030e00 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.863+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f88103190 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.865+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2f800305a0 con 0x7f2f881082e0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.865+0000 7f2f777fe640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 0x7f2f700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.866+0000 7f2f8659c640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 0x7f2f700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.866+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f2f8002e030 con 0x7f2f881082e0 2024-09-19T05:41:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.866+0000 7f2f8659c640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 0x7f2f700a5700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009b60 tx=0x7f2f78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:43.868+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f8004b060 con 0x7f2f881082e0 2024-09-19T05:41:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.084+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f8810a2c0 con 0x7f2f881082e0 2024-09-19T05:41:44.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.085+0000 7f2f777fe640 1 -- 172.21.15.38:0/4178702502 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f2f800bdd10 con 0x7f2f881082e0 2024-09-19T05:41:44.087 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:41:44.087 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:41:44.087 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 msgr2=0x7f2f700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 0x7f2f700a5700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009b60 tx=0x7f2f78005f90 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 msgr2=0x7f2f88101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005cc0 tx=0x7f2f800093d0 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 shutdown_connections 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f2f700a3240 0x7f2f700a5700 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f881082e0 0x7f2f88101490 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f88103760 0x7f2f88100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 --2- 172.21.15.38:0/4178702502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f88102db0 0x7f2f88100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 >> 172.21.15.38:0/4178702502 conn(0x7f2f880fc7d0 msgr2=0x7f2f88106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 shutdown_connections 2024-09-19T05:41:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:44.087+0000 7f2f8c8c7640 1 -- 172.21.15.38:0/4178702502 wait complete. 2024-09-19T05:41:44.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:44 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4178702502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:44.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:44 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/4178702502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:44.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:44 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/4178702502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:41:44.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:39:02.544314+0000', 'modified': '2024-09-19T05:41:30.491887+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24496, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14547': {'gid': 14547, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6837/4064878755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6836', 'nonce': 4064878755}, {'type': 'v1', 'addr': '172.21.15.38:6837', 'nonce': 4064878755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6839/2963619728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6838', 'nonce': 2963619728}, {'type': 'v1', 'addr': '172.21.15.38:6839', 'nonce': 2963619728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6841/3698066275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6840', 'nonce': 3698066275}, {'type': 'v1', 'addr': '172.21.15.38:6841', 'nonce': 3698066275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6835/3758113926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3758113926}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3758113926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/2514682242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2514682242}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 2514682242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.47:6837/2962930600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2962930600}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2962930600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.179:6835/994628887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 994628887}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 994628887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.179:6837/164318179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 164318179}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 164318179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.47:6839/1610791508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 1610791508}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 1610791508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.179:6839/2081935059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2081935059}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2081935059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24455, 24481, 24475, 24485, 24496]} 2024-09-19T05:41:44.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T05:41:44.830 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T05:41:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:45.536 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:45 smithi038 ceph-mon[31227]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:41:45.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 -- 172.21.15.38:0/3624731889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34102ef0 msgr2=0x7fdc3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/3624731889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34102ef0 0x7fdc3410da40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009900 tx=0x7fdc1c02a480 comp rx=0 tx=0).stop 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 -- 172.21.15.38:0/3624731889 shutdown_connections 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/3624731889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34102ef0 0x7fdc3410da40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/3624731889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102570 0x7fdc341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/3624731889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34100e20 0x7fdc34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 -- 172.21.15.38:0/3624731889 >> 172.21.15.38:0/3624731889 conn(0x7fdc340fc7d0 msgr2=0x7fdc340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 -- 172.21.15.38:0/3624731889 shutdown_connections 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 -- 172.21.15.38:0/3624731889 wait complete. 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.617+0000 7fdc3af6d640 1 Processor -- start 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 -- start start 2024-09-19T05:41:45.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 0x7fdc34199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102ef0 0x7fdc34199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34110f80 con 0x7fdc34100e20 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34110e00 con 0x7fdc34102ef0 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3af6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34111100 con 0x7fdc34102570 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35170/0 (socket says 172.21.15.38:35170) 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 -- 172.21.15.38:0/1795371766 learned_addr learned my addr 172.21.15.38:0/1795371766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102ef0 0x7fdc34199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc33fff640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 0x7fdc34199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 -- 172.21.15.38:0/1795371766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 msgr2=0x7fdc34199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 0x7fdc34199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 -- 172.21.15.38:0/1795371766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102ef0 msgr2=0x7fdc34199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102ef0 0x7fdc34199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 -- 172.21.15.38:0/1795371766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc341a47d0 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc33fff640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 0x7fdc34199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.618+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d770 tx=0x7fdc2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.619+0000 7fdc31ffb640 1 -- 172.21.15.38:0/1795371766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c017070 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.619+0000 7fdc31ffb640 1 -- 172.21.15.38:0/1795371766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c005500 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.619+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc341a4ac0 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.619+0000 7fdc31ffb640 1 -- 172.21.15.38:0/1795371766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004e00 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.619+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a4fb0 con 0x7fdc34100e20 2024-09-19T05:41:45.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc31ffb640 1 -- 172.21.15.38:0/1795371766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fdc2c004fa0 con 0x7fdc34100e20 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 Processor -- start 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 -- start start 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a05319fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531aabb0 0x55a0531acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a053106090 con 0x55a0531a02f0 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a0530ada70 con 0x55a05319f9d0 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc3d346740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a053102050 con 0x55a0531aabb0 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc38d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc38d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35182/0 (socket says 172.21.15.38:35182) 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.622+0000 7fdc38d09640 1 -- 172.21.15.38:0/147629523 learned_addr learned my addr 172.21.15.38:0/147629523 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:45.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc3950a640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a05319fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc33fff640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531aabb0 0x55a0531acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc31ffb640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 0x7fdc200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 -- 172.21.15.38:0/147629523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531aabb0 msgr2=0x55a0531acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531aabb0 0x55a0531acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 -- 172.21.15.38:0/147629523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 msgr2=0x55a05319fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a05319fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc31ffb640 1 -- 172.21.15.38:0/1795371766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fdc2c0cf750 con 0x7fdc34100e20 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 -- 172.21.15.38:0/147629523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0530ba130 con 0x55a0531a02f0 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc3950a640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a05319fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:45.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 0x7fdc200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc38d09640 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0cce90 tx=0x7fdc2c0862f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e735c4535a824c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc2affd640 1 -- 172.21.15.38:0/147629523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c020070 con 0x55a0531a02f0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc2affd640 1 -- 172.21.15.38:0/147629523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c0046c0 con 0x55a0531a02f0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.623+0000 7fdc2affd640 1 -- 172.21.15.38:0/147629523 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0a4620 con 0x55a0531a02f0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- 172.21.15.38:0/147629523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 msgr2=0x55a0531aa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 0x7fdc200a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc3406b050 tx=0x7fdc1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0cce90 tx=0x7fdc2c0862f0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- 172.21.15.38:0/147629523 shutdown_connections 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531aabb0 0x55a0531acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531aa670 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 --2- 172.21.15.38:0/147629523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a05319fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- 172.21.15.38:0/147629523 >> 172.21.15.38:0/147629523 conn(0x55a05319bf60 msgr2=0x55a05319c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- 172.21.15.38:0/147629523 shutdown_connections 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- 172.21.15.38:0/147629523 wait complete. 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 -- start start 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.624+0000 7fdc3d346740 1 Processor -- start 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0531ad720 con 0x55a0531aabb0 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a0530c0f20 con 0x55a05319f9d0 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a0531ad8a0 con 0x55a0531a02f0 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3d346740 10 client.0 _ll_register_callbacks cb 0x55a05308a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:45.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc38d09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35188/0 (socket says 172.21.15.38:35188) 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 -- 172.21.15.38:0/1151714273 learned_addr learned my addr 172.21.15.38:0/1151714273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc33fff640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 -- 172.21.15.38:0/1151714273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 msgr2=0x55a0531a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 -- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 msgr2=0x55a0531aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc3950a640 1 -- 172.21.15.38:0/1151714273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a052e772b0 con 0x55a0531aabb0 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc33fff640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.625+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc3950a640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005ec0 tx=0x7fdc1c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f6058f51f87166 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c035040 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fdc1c031950 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a053113600 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc1c05a050 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc3d346740 10 client.15357 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.626+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0531b9040 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a053113600 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc1c048b00 con 0x55a0531aabb0 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 10 client.15357 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 10 client.15357 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 20 client.15357 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a0531bb720 0x55a0531bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 4 client.15357 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05310db10 con 0x55a0531bb720 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.627+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a0531bb720 0x55a0531bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.628+0000 7fdc38d09640 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a0531bb720 0x55a0531bcb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=159f6631515f36a4 server_cookie=6a05f321b124212e in_seq=0 out_seq=0 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.628+0000 7fdc13fff640 10 client.15357 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.628+0000 7fdc13fff640 1 -- 172.21.15.38:0/1151714273 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a05310db10 con 0x55a0531bb720 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.628+0000 7fdc13fff640 10 client.15357 handle_command_reply: tid=0 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.628+0000 7fdc3d346740 1 client.15357 shutdown 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 msgr2=0x55a0531a7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005ec0 tx=0x7fdc1c02a480 comp rx=0 tx=0).stop 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 shutdown_connections 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a0531bb720 0x55a0531bcb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531a7cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 --2- 172.21.15.38:0/1151714273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 >> 172.21.15.38:0/1151714273 conn(0x55a05319bf60 msgr2=0x55a0531a1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc13fff640 0 client.15357 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 shutdown_connections 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 1 -- 172.21.15.38:0/1151714273 wait complete. 2024-09-19T05:41:45.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.629+0000 7fdc3d346740 20 client.15357 trim_cache size 0 max 16384 2024-09-19T05:41:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 Processor -- start 2024-09-19T05:41:45.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 -- start start 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a05319f9d0 0x55a0531bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a0531aabb0 0x55a0531a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0531baed0 con 0x55a0531a02f0 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a0531bb5a0 con 0x55a0531aabb0 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3d346740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a0531bbc70 con 0x55a05319f9d0 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35198/0 (socket says 172.21.15.38:35198) 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc33fff640 1 -- 172.21.15.38:0/32480513 learned_addr learned my addr 172.21.15.38:0/32480513 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.647+0000 7fdc3950a640 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a0531aabb0 0x55a0531a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc38d09640 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a05319f9d0 0x55a0531bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 -- 172.21.15.38:0/32480513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a05319f9d0 msgr2=0x55a0531bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a05319f9d0 0x55a0531bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 -- 172.21.15.38:0/32480513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a0531aabb0 msgr2=0x55a0531a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a0531aabb0 0x55a0531a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 -- 172.21.15.38:0/32480513 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05319bd90 con 0x55a0531a02f0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc33fff640 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc340fece0 tx=0x7fdc24004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320fedcc9f4b1943 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc2a7fc640 1 -- 172.21.15.38:0/32480513 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24012450 con 0x55a0531a02f0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc2a7fc640 1 -- 172.21.15.38:0/32480513 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc24012ad0 con 0x55a0531a02f0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.648+0000 7fdc2a7fc640 1 -- 172.21.15.38:0/32480513 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2401cc30 con 0x55a0531a02f0 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- 172.21.15.38:0/32480513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 msgr2=0x55a0531bc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc340fece0 tx=0x7fdc24004870 comp rx=0 tx=0).stop 2024-09-19T05:41:45.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- 172.21.15.38:0/32480513 shutdown_connections 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a0531aabb0 0x55a0531a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531a02f0 0x55a0531bc4c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 --2- 172.21.15.38:0/32480513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a05319f9d0 0x55a0531bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- 172.21.15.38:0/32480513 >> 172.21.15.38:0/32480513 conn(0x55a05319bf60 msgr2=0x55a0531ba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- 172.21.15.38:0/32480513 shutdown_connections 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- 172.21.15.38:0/32480513 wait complete. 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 -- start start 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.649+0000 7fdc3d346740 1 Processor -- start 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a0531bca10 con 0x55a0531aabb0 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a0531bc340 con 0x55a05319f9d0 2024-09-19T05:41:45.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a0531bd0e0 con 0x55a0531a02f0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3d346740 10 client.0 _ll_register_callbacks cb 0x7fdc1c005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35208/0 (socket says 172.21.15.38:35208) 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 -- 172.21.15.38:0/828574174 learned_addr learned my addr 172.21.15.38:0/828574174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc33fff640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc38d09640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 -- 172.21.15.38:0/828574174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 msgr2=0x55a0531bde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531bde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 -- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 msgr2=0x55a0531c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 -- 172.21.15.38:0/828574174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0531bec00 con 0x55a0531aabb0 2024-09-19T05:41:45.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc33fff640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531bde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc3950a640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005d00 tx=0x7fdc1c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c9dc6ec02bc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.650+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c035040 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fdc1c005440 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc24012ad0 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc1c067050 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 10 client.15363 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a053224f20 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc24012ad0 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc1c004990 con 0x55a0531aabb0 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 10 client.15363 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:41:45.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 20 client.15363 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a053229ac0 0x55a0532287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 4 client.15363 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.651+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 --> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] -- command(tid 0: {"prefix": "status"}) -- 0x55a05308a0e0 con 0x55a053229ac0 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc38d09640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a053229ac0 0x55a0532287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc38d09640 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a053229ac0 0x55a0532287d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=adbcc1e1cfca776 server_cookie=ea348bb396b0fec2 in_seq=0 out_seq=0 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc13fff640 10 client.15363 ms_handle_connect on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc13fff640 1 -- 172.21.15.38:0/828574174 <== mds.4 v2:172.21.15.47:6836/2962930600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a05308a0e0 con 0x55a053229ac0 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc13fff640 10 client.15363 handle_command_reply: tid=0 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 client.15363 shutdown 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 msgr2=0x55a0531be3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005d00 tx=0x7fdc1c040000 comp rx=0 tx=0).stop 2024-09-19T05:41:45.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 shutdown_connections 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:6836/2962930600,v1:172.21.15.47:6837/2962930600] conn(0x55a053229ac0 0x55a0532287d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a0531aabb0 0x55a0531be3d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0531a02f0 0x55a0531bde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 --2- 172.21.15.38:0/828574174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a05319f9d0 0x55a0531c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 >> 172.21.15.38:0/828574174 conn(0x55a05319bf60 msgr2=0x55a0531ba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.652+0000 7fdc13fff640 0 client.15363 ms_handle_reset on v2:172.21.15.47:6836/2962930600 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 shutdown_connections 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3d346740 1 -- 172.21.15.38:0/828574174 wait complete. 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3d346740 20 client.15363 trim_cache size 0 max 16384 2024-09-19T05:41:45.668 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 4, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24455, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 118.554458739, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 28, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 185.21253028000001 2024-09-19T05:41:45.669 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 msgr2=0x7fdc200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 0x7fdc200a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc3406b050 tx=0x7fdc1c009260 comp rx=0 tx=0).stop 2024-09-19T05:41:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 msgr2=0x7fdc341993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:45.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d770 tx=0x7fdc2c00dc30 comp rx=0 tx=0).stop 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 shutdown_connections 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fdc200a3240 0x7fdc200a5700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc34102ef0 0x7fdc34199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc34102570 0x7fdc34199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 --2- 172.21.15.38:0/1795371766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc34100e20 0x7fdc341993d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 >> 172.21.15.38:0/1795371766 conn(0x7fdc340fc7d0 msgr2=0x7fdc341068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 shutdown_connections 2024-09-19T05:41:45.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:45.653+0000 7fdc3af6d640 1 -- 172.21.15.38:0/1795371766 wait complete. 2024-09-19T05:41:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:45 smithi179 ceph-mon[35781]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:41:45.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:45 smithi047 ceph-mon[35327]: pgmap v368: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:41:46.370 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T05:41:46.632 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3825662075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 msgr2=0x7f7f0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3825662075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0410ea90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009ca0 tx=0x7f7efc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3825662075 shutdown_connections 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3825662075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0410ea90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3825662075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 0x7f7f041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3825662075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 0x7f7f041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3825662075 >> 172.21.15.38:0/3825662075 conn(0x7f7f040fd8c0 msgr2=0x7f7f040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3825662075 shutdown_connections 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3825662075 wait complete. 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 Processor -- start 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.111+0000 7f7f0abc3640 1 -- start start 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 0x7f7f04198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 0x7f7f041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04111f90 con 0x7f7f04106f10 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04111e10 con 0x7f7f04102790 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0abc3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112110 con 0x7f7f04101de0 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35226/0 (socket says 172.21.15.38:35226) 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 -- 172.21.15.38:0/3125908432 learned_addr learned my addr 172.21.15.38:0/3125908432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7efbfff640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 0x7f7f041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f0895f640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 0x7f7f04198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 -- 172.21.15.38:0/3125908432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 msgr2=0x7f7f04198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 0x7f7f04198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 -- 172.21.15.38:0/3125908432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 msgr2=0x7f7f041989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 0x7f7f041989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.112+0000 7f7f09160640 1 -- 172.21.15.38:0/3125908432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0419d420 con 0x7f7f04106f10 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7f09160640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7efc031a20 tx=0x7f7efc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7ef9ffb640 1 -- 172.21.15.38:0/3125908432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003b70 con 0x7f7f04106f10 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7ef9ffb640 1 -- 172.21.15.38:0/3125908432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7efc003d10 con 0x7f7f04106f10 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7ef9ffb640 1 -- 172.21.15.38:0/3125908432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc02fc80 con 0x7f7f04106f10 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0419d6b0 con 0x7f7f04106f10 2024-09-19T05:41:47.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.113+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041a4ff0 con 0x7f7f04106f10 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.115+0000 7f7ef9ffb640 1 -- 172.21.15.38:0/3125908432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7efc0046a0 con 0x7f7f04106f10 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 Processor -- start 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 -- start start 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56082c9ba090 con 0x56082ca539d0 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56082c961a70 con 0x56082ca542f0 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56082c9b6050 con 0x56082ca5ebb0 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35228/0 (socket says 172.21.15.38:35228) 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7efbfff640 1 -- 172.21.15.38:0/596838037 learned_addr learned my addr 172.21.15.38:0/596838037 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7ef9ffb640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 0x7f7eec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.116+0000 7f7f0895f640 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 -- 172.21.15.38:0/596838037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 msgr2=0x56082ca60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 -- 172.21.15.38:0/596838037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 msgr2=0x56082ca5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7f09160640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 0x7f7eec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 -- 172.21.15.38:0/596838037 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082c96e130 con 0x56082ca539d0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7ef9ffb640 1 -- 172.21.15.38:0/3125908432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7efc0f0e80 con 0x7f7f04106f10 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7efbfff640 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40058a0 tx=0x7f7ef40127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dac9dcf48a7e11b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7eeaffd640 1 -- 172.21.15.38:0/596838037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40120d0 con 0x56082ca539d0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7eeaffd640 1 -- 172.21.15.38:0/596838037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4016070 con 0x56082ca539d0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7eeaffd640 1 -- 172.21.15.38:0/596838037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4010870 con 0x56082ca539d0 2024-09-19T05:41:47.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.117+0000 7f7f09160640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 0x7f7eec0a5780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0b9ae0 tx=0x7f7efc033040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/596838037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 msgr2=0x56082ca53db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40058a0 tx=0x7f7ef40127c0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/596838037 shutdown_connections 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/596838037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca53db0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/596838037 >> 172.21.15.38:0/596838037 conn(0x56082ca4ff60 msgr2=0x56082ca50340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/596838037 shutdown_connections 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/596838037 wait complete. 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 -- start start 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.118+0000 7f7f0cf9c740 1 Processor -- start 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca539d0 0x56082ca5e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca5b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56082ca61780 con 0x56082ca5ebb0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56082c974f20 con 0x56082ca539d0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56082ca61900 con 0x56082ca542f0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0cf9c740 10 client.0 _ll_register_callbacks cb 0x56082c93e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35238/0 (socket says 172.21.15.38:35238) 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 -- 172.21.15.38:0/3202232812 learned_addr learned my addr 172.21.15.38:0/3202232812 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca539d0 0x56082ca5e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7f0895f640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca5b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 -- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 msgr2=0x56082ca5b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca5b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 -- 172.21.15.38:0/3202232812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca539d0 msgr2=0x56082ca5e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca539d0 0x56082ca5e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.119+0000 7f7efbfff640 1 -- 172.21.15.38:0/3202232812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082c72b2b0 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7efbfff640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7ef401bef0 tx=0x7f7ef4010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af196c449fd3420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401b470 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7ef401b610 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef4016070 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ef401b810 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 10 client.15375 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56082ca6d0f0 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4016070 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7ef40035f0 con 0x56082ca5ebb0 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 10 client.15375 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 10 client.15375 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 20 client.15375 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:47.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082ca6f300 0x56082ca70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 4 client.15375 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.120+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082c9c7600 con 0x56082ca6f300 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.121+0000 7f7f0895f640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082ca6f300 0x56082ca70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.121+0000 7f7f0895f640 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082ca6f300 0x56082ca70770 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2d29c02d6b9146d server_cookie=2e9c63858022c833 in_seq=0 out_seq=0 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.121+0000 7f7ee3fff640 10 client.15375 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7ee3fff640 1 -- 172.21.15.38:0/3202232812 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56082c9c7600 con 0x56082ca6f300 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7ee3fff640 10 client.15375 handle_command_reply: tid=0 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 client.15375 shutdown 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 msgr2=0x56082ca5bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7ef401bef0 tx=0x7f7ef4010430 comp rx=0 tx=0).stop 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 shutdown_connections 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082ca6f300 0x56082ca70770 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca5ebb0 0x56082ca5bd30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca5b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/3202232812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca539d0 0x56082ca5e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7ee3fff640 0 client.15375 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 >> 172.21.15.38:0/3202232812 conn(0x56082ca4ff60 msgr2=0x56082ca55c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 shutdown_connections 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/3202232812 wait complete. 2024-09-19T05:41:47.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.122+0000 7f7f0cf9c740 20 client.15375 trim_cache size 0 max 16384 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.140+0000 7f7f0cf9c740 1 Processor -- start 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.140+0000 7f7f0cf9c740 1 -- start start 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.140+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.140+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56082ca6eab0 con 0x56082ca539d0 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56082ca6f180 con 0x56082ca542f0 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56082ca6f850 con 0x56082ca5ebb0 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35246/0 (socket says 172.21.15.38:35246) 2024-09-19T05:41:47.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 -- 172.21.15.38:0/1309803295 learned_addr learned my addr 172.21.15.38:0/1309803295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 -- 172.21.15.38:0/1309803295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 msgr2=0x56082ca5a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 -- 172.21.15.38:0/1309803295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 msgr2=0x56082ca5a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 -- 172.21.15.38:0/1309803295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082ca5e0c0 con 0x56082ca539d0 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4015040 tx=0x7f7ef400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ea13c6f1c9492d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7eea7fc640 1 -- 172.21.15.38:0/1309803295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401b670 con 0x56082ca539d0 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7eea7fc640 1 -- 172.21.15.38:0/1309803295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4016040 con 0x56082ca539d0 2024-09-19T05:41:47.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.141+0000 7f7eea7fc640 1 -- 172.21.15.38:0/1309803295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003c00 con 0x56082ca539d0 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1309803295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 msgr2=0x56082ca70770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4015040 tx=0x7f7ef400b390 comp rx=0 tx=0).stop 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1309803295 shutdown_connections 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca5ebb0 0x56082ca5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca542f0 0x56082ca5a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1309803295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca70770 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1309803295 >> 172.21.15.38:0/1309803295 conn(0x56082ca4ff60 msgr2=0x56082ca4ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1309803295 shutdown_connections 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1309803295 wait complete. 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.142+0000 7f7f0cf9c740 1 -- start start 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 Processor -- start 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca5ebb0 0x56082cad19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56082ca705f0 con 0x56082ca539d0 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56082ca6ff20 con 0x56082ca5ebb0 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56082ca70cc0 con 0x56082ca542f0 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0cf9c740 10 client.0 _ll_register_callbacks cb 0x7f7ef400bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35250/0 (socket says 172.21.15.38:35250) 2024-09-19T05:41:47.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 -- 172.21.15.38:0/1790793267 learned_addr learned my addr 172.21.15.38:0/1790793267 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 -- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 msgr2=0x56082ca745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 -- 172.21.15.38:0/1790793267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca5ebb0 msgr2=0x56082cad19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca5ebb0 0x56082cad19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.143+0000 7f7efbfff640 1 -- 172.21.15.38:0/1790793267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082cad1f90 con 0x56082ca539d0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7efbfff640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4003a10 tx=0x7f7ef40057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccefad9fe34dfe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003800 con 0x56082ca539d0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7ef401b9b0 con 0x56082ca539d0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef4016040 con 0x56082ca539d0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ef4044050 con 0x56082ca539d0 2024-09-19T05:41:47.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7f0cf9c740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56082cad9340 con 0x56082ca539d0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4016040 con 0x56082ca539d0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7ef402a400 con 0x56082ca539d0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.144+0000 7f7f0cf9c740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 10 client.15381 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 20 client.15381 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082caddae0 0x56082cadc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 4 client.15381 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 --> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] -- command(tid 0: {"prefix": "status"}) -- 0x56082c9c1b10 con 0x56082caddae0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082caddae0 0x56082cadc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0895f640 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082caddae0 0x56082cadc7f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f907b97af0296022 server_cookie=cdcc6828966638cb in_seq=0 out_seq=0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7ee3fff640 10 client.15381 ms_handle_connect on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7ee3fff640 1 -- 172.21.15.38:0/1790793267 <== mds.3 v2:172.21.15.179:6834/994628887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56082c9c1b10 con 0x56082caddae0 2024-09-19T05:41:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7ee3fff640 10 client.15381 handle_command_reply: tid=0 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.145+0000 7f7f0cf9c740 1 client.15381 shutdown 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 msgr2=0x56082ca74070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4003a10 tx=0x7f7ef40057e0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 shutdown_connections 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56082ca5ebb0 0x56082cad19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56082ca542f0 0x56082ca745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.179:6834/994628887,v1:172.21.15.179:6835/994628887] conn(0x56082caddae0 0x56082cadc7f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 --2- 172.21.15.38:0/1790793267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56082ca539d0 0x56082ca74070 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7ee3fff640 0 client.15381 ms_handle_reset on v2:172.21.15.179:6834/994628887 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 >> 172.21.15.38:0/1790793267 conn(0x56082ca4ff60 msgr2=0x56082ca59ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 shutdown_connections 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 1 -- 172.21.15.38:0/1790793267 wait complete. 2024-09-19T05:41:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0cf9c740 20 client.15381 trim_cache size 0 max 16384 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 3, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24475, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 121.05488384500001, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 28, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T05:41:47.154 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 191.93170049299999 2024-09-19T05:41:47.155 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 msgr2=0x7f7eec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 0x7f7eec0a5780 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0b9ae0 tx=0x7f7efc033040 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 msgr2=0x7f7f0419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.146+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7efc031a20 tx=0x7f7efc031a50 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 shutdown_connections 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7eec0a32c0 0x7f7eec0a5780 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f04106f10 0x7f7f0419cd40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f04102790 0x7f7f041989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 --2- 172.21.15.38:0/3125908432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f04101de0 0x7f7f04198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 >> 172.21.15.38:0/3125908432 conn(0x7f7f040fd8c0 msgr2=0x7f7f040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 shutdown_connections 2024-09-19T05:41:47.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:47.147+0000 7f7f0abc3640 1 -- 172.21.15.38:0/3125908432 wait complete. 2024-09-19T05:41:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:47 smithi038 ceph-mon[31227]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:41:47.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:47 smithi179 ceph-mon[35781]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:41:47.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:47 smithi047 ceph-mon[35327]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:41:47.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:41:48.130 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 -- 172.21.15.38:0/2515169673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 msgr2=0x7f7fec105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/2515169673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec105e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009900 tx=0x7f7fdc02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 -- 172.21.15.38:0/2515169673 shutdown_connections 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/2515169673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101bc0 0x7f7fec10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/2515169673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101240 0x7f7fec101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/2515169673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec105e70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 -- 172.21.15.38:0/2515169673 >> 172.21.15.38:0/2515169673 conn(0x7f7fec077210 msgr2=0x7f7fec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 -- 172.21.15.38:0/2515169673 shutdown_connections 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.589+0000 7f7ff23bf640 1 -- 172.21.15.38:0/2515169673 wait complete. 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 Processor -- start 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 -- start start 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101240 0x7f7fec0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 0x7f7fec0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec110f70 con 0x7f7fec105a90 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec110df0 con 0x7f7fec101bc0 2024-09-19T05:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff23bf640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec1110f0 con 0x7f7fec101240 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff095c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff095c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36706/0 (socket says 172.21.15.38:36706) 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7ff095c640 1 -- 172.21.15.38:0/3979510774 learned_addr learned my addr 172.21.15.38:0/3979510774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7febfff640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101240 0x7f7fec0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.590+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 0x7f7fec0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 -- 172.21.15.38:0/3979510774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101240 msgr2=0x7f7fec0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101240 0x7f7fec0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 -- 172.21.15.38:0/3979510774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 msgr2=0x7f7fec0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 0x7f7fec0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 -- 172.21.15.38:0/3979510774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec1a6860 con 0x7f7fec105a90 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 0x7f7fec0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff095c640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400cce0 tx=0x7f7fe400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7fe97fa640 1 -- 172.21.15.38:0/3979510774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400f450 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7fe97fa640 1 -- 172.21.15.38:0/3979510774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe400fad0 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec1a6b50 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7fe97fa640 1 -- 172.21.15.38:0/3979510774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4018c10 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.591+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a7090 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7fe97fa640 1 -- 172.21.15.38:0/3979510774 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7fe400f5f0 con 0x7f7fec105a90 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 Processor -- start 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 -- start start 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36deddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36dee2f0 0x55ae36df8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae36d54090 con 0x55ae36df8bb0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ae36cfba70 con 0x55ae36dee2f0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7ff4798740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae36d50050 con 0x55ae36ded9d0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7feb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7feb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36722/0 (socket says 172.21.15.38:36722) 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7feb7fe640 1 -- 172.21.15.38:0/1045865880 learned_addr learned my addr 172.21.15.38:0/1045865880 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36deddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.594+0000 7f7febfff640 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36dee2f0 0x55ae36df8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7fe97fa640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 0x7f7fd40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 -- 172.21.15.38:0/1045865880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 msgr2=0x55ae36deddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36deddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 -- 172.21.15.38:0/1045865880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36dee2f0 msgr2=0x55ae36df8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36dee2f0 0x55ae36df8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 -- 172.21.15.38:0/1045865880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae36d08130 con 0x55ae36df8bb0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7ff095c640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 0x7f7fd40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0012880 tx=0x7f7fe000d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb98ad0b5afacc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7fe97fa640 1 -- 172.21.15.38:0/3979510774 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7fe40d0340 con 0x7f7fec105a90 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7fd27fc640 1 -- 172.21.15.38:0/1045865880 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000aa60 con 0x55ae36df8bb0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7fd27fc640 1 -- 172.21.15.38:0/1045865880 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe0005500 con 0x55ae36df8bb0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.595+0000 7f7fd27fc640 1 -- 172.21.15.38:0/1045865880 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0004c60 con 0x55ae36df8bb0 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- 172.21.15.38:0/1045865880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 msgr2=0x55ae36dfafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff095c640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 0x7f7fd40a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7fe409c2a0 tx=0x7f7fe406b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0012880 tx=0x7f7fe000d440 comp rx=0 tx=0).stop 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- 172.21.15.38:0/1045865880 shutdown_connections 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36df8bb0 0x55ae36dfafa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36dee2f0 0x55ae36df8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 --2- 172.21.15.38:0/1045865880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36deddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- 172.21.15.38:0/1045865880 >> 172.21.15.38:0/1045865880 conn(0x55ae36de9f60 msgr2=0x55ae36dea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- 172.21.15.38:0/1045865880 shutdown_connections 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- 172.21.15.38:0/1045865880 wait complete. 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 -- start start 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.596+0000 7f7ff4798740 1 Processor -- start 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36df83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36df8bb0 0x55ae36df5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae36dfb780 con 0x55ae36dee2f0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ae36d0ef20 con 0x55ae36df8bb0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae36dfb900 con 0x55ae36ded9d0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7ff4798740 10 client.0 _ll_register_callbacks cb 0x55ae36cd8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36732/0 (socket says 172.21.15.38:36732) 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 -- 172.21.15.38:0/922868339 learned_addr learned my addr 172.21.15.38:0/922868339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36df8bb0 0x55ae36df5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36df83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 -- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 msgr2=0x55ae36df83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36df83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 -- 172.21.15.38:0/922868339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36df8bb0 msgr2=0x55ae36df5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36df8bb0 0x55ae36df5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7febfff640 1 -- 172.21.15.38:0/922868339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae36ac52b0 con 0x55ae36dee2f0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.597+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36df83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7febfff640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02a910 tx=0x7f7fdc004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567a909eb7708a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc035040 con 0x55ae36dee2f0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7fdc003bc0 con 0x55ae36dee2f0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe0005500 con 0x55ae36dee2f0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc003dc0 con 0x55ae36dee2f0 2024-09-19T05:41:48.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7ff4798740 10 client.15393 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae36e07010 con 0x55ae36dee2f0 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe0005500 con 0x55ae36dee2f0 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7fdc03d780 con 0x55ae36dee2f0 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.598+0000 7f7ff4798740 10 client.15393 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7ff4798740 10 client.15393 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7ff4798740 20 client.15393 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7ff4798740 4 client.15393 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae36d61600 con 0x55ae36e09610 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e0aa80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc15e79a53dc7467 server_cookie=4132c0ab51447be7 in_seq=0 out_seq=0 2024-09-19T05:41:48.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.599+0000 7f7fcb7fe640 10 client.15393 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/922868339 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae36d61600 con 0x55ae36e09610 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7fcb7fe640 10 client.15393 handle_command_reply: tid=0 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 client.15393 shutdown 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 msgr2=0x55ae36df5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02a910 tx=0x7f7fdc004580 comp rx=0 tx=0).stop 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 shutdown_connections 2024-09-19T05:41:48.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e0aa80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7fcb7fe640 0 client.15393 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36df8bb0 0x55ae36df5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36df5710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 --2- 172.21.15.38:0/922868339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36ded9d0 0x55ae36df83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 >> 172.21.15.38:0/922868339 conn(0x55ae36de9f60 msgr2=0x55ae36defc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 shutdown_connections 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 1 -- 172.21.15.38:0/922868339 wait complete. 2024-09-19T05:41:48.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.600+0000 7f7ff4798740 20 client.15393 trim_cache size 0 max 16384 2024-09-19T05:41:48.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 Processor -- start 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 -- start start 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36df7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e08dc0 con 0x55ae36dee2f0 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e09490 con 0x55ae36ded9d0 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7ff4798740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e09b60 con 0x55ae36df8bb0 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.618+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36736/0 (socket says 172.21.15.38:36736) 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 -- 172.21.15.38:0/2410506386 learned_addr learned my addr 172.21.15.38:0/2410506386 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:48.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36df7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 -- 172.21.15.38:0/2410506386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 msgr2=0x55ae36df7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36df7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 -- 172.21.15.38:0/2410506386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 msgr2=0x55ae36e0aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 -- 172.21.15.38:0/2410506386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae36df7fe0 con 0x55ae36dee2f0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7febfff640 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006010 tx=0x7f7fdc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19f88f39b3587fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7fd1ffb640 1 -- 172.21.15.38:0/2410506386 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc035040 con 0x55ae36dee2f0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7fd1ffb640 1 -- 172.21.15.38:0/2410506386 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc041070 con 0x55ae36dee2f0 2024-09-19T05:41:48.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.619+0000 7f7fd1ffb640 1 -- 172.21.15.38:0/2410506386 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc005310 con 0x55ae36dee2f0 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- 172.21.15.38:0/2410506386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 msgr2=0x55ae36e08f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006010 tx=0x7f7fdc0035f0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- 172.21.15.38:0/2410506386 shutdown_connections 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36df7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e08f40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2410506386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- 172.21.15.38:0/2410506386 >> 172.21.15.38:0/2410506386 conn(0x55ae36de9f60 msgr2=0x55ae36df6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- 172.21.15.38:0/2410506386 shutdown_connections 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- 172.21.15.38:0/2410506386 wait complete. 2024-09-19T05:41:48.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 -- start start 2024-09-19T05:41:48.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.620+0000 7f7ff4798740 1 Processor -- start 2024-09-19T05:41:48.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36e0f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e0a900 con 0x55ae36dee2f0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e0a230 con 0x55ae36ded9d0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae36e0afd0 con 0x55ae36df8bb0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7ff4798740 10 client.0 _ll_register_callbacks cb 0x7f7fdc004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36742/0 (socket says 172.21.15.38:36742) 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 -- 172.21.15.38:0/2800836996 learned_addr learned my addr 172.21.15.38:0/2800836996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36e0f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 -- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 msgr2=0x55ae36e0f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36e0f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 -- 172.21.15.38:0/2800836996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 msgr2=0x55ae36e0eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 -- 172.21.15.38:0/2800836996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae36e72d50 con 0x55ae36dee2f0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7febfff640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009850 tx=0x7f7fdc003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd6c999ff42e368 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc035040 con 0x55ae36dee2f0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.621+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7fdc005290 con 0x55ae36dee2f0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc041070 con 0x55ae36dee2f0 2024-09-19T05:41:48.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc0039c0 con 0x55ae36dee2f0 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 10 client.15399 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae36e73f80 con 0x55ae36dee2f0 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc041070 con 0x55ae36dee2f0 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7fdc0455f0 con 0x55ae36dee2f0 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 10 client.15399 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 20 client.15399 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 4 client.15399 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.622+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 --> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] -- command(tid 0: {"prefix": "status"}) -- 0x55ae36d5bb10 con 0x55ae36e09610 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:48.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7fcb7fe640 10 client.15399 ms_handle_connect on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7feb7fe640 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e77270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b224777da7b286b2 server_cookie=2a682dc01187284c in_seq=0 out_seq=0 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7fcb7fe640 1 -- 172.21.15.38:0/2800836996 <== mds.2 v2:172.21.15.179:6836/164318179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ae36d5bb10 con 0x55ae36e09610 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7fcb7fe640 10 client.15399 handle_command_reply: tid=0 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 client.15399 shutdown 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 msgr2=0x55ae36e0eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009850 tx=0x7f7fdc003cf0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 shutdown_connections 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:6836/164318179,v1:172.21.15.179:6837/164318179] conn(0x55ae36e09610 0x55ae36e77270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36df8bb0 0x55ae36e0f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae36dee2f0 0x55ae36e0eff0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 --2- 172.21.15.38:0/2800836996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ae36ded9d0 0x55ae36e0eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7fcb7fe640 0 client.15399 ms_handle_reset on v2:172.21.15.179:6836/164318179 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.623+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 >> 172.21.15.38:0/2800836996 conn(0x55ae36de9f60 msgr2=0x55ae36de9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 shutdown_connections 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff4798740 1 -- 172.21.15.38:0/2800836996 wait complete. 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff4798740 20 client.15399 trim_cache size 0 max 16384 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:48.632 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24481, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 123.54008213500001, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 28, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 190.79492901699999 2024-09-19T05:41:48.633 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:48.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 msgr2=0x7f7fd40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 0x7f7fd40a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7fe409c2a0 tx=0x7f7fe406b2e0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 msgr2=0x7f7fec0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:48.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400cce0 tx=0x7f7fe400a620 comp rx=0 tx=0).stop 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 shutdown_connections 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7fd40a32c0 0x7f7fd40a5780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fec105a90 0x7f7fec0fd110 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fec101bc0 0x7f7fec0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 --2- 172.21.15.38:0/3979510774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fec101240 0x7f7fec0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 >> 172.21.15.38:0/3979510774 conn(0x7f7fec077210 msgr2=0x7f7fec100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 shutdown_connections 2024-09-19T05:41:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:48.624+0000 7f7ff23bf640 1 -- 172.21.15.38:0/3979510774 wait complete. 2024-09-19T05:41:49.341 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:41:49.596 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:49.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:49 smithi038 ceph-mon[31227]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:41:49.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:49 smithi179 ceph-mon[35781]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:41:49.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:49 smithi047 ceph-mon[35327]: pgmap v370: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 -- 172.21.15.38:0/1596126068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c101de0 msgr2=0x7fbf0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 --2- 172.21.15.38:0/1596126068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1021c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ca0 tx=0x7fbefc02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 -- 172.21.15.38:0/1596126068 shutdown_connections 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 --2- 172.21.15.38:0/1596126068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c106f10 0x7fbf0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 --2- 172.21.15.38:0/1596126068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c102790 0x7fbf0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 --2- 172.21.15.38:0/1596126068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1021c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 -- 172.21.15.38:0/1596126068 >> 172.21.15.38:0/1596126068 conn(0x7fbf0c0fd8c0 msgr2=0x7fbf0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 -- 172.21.15.38:0/1596126068 shutdown_connections 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 -- 172.21.15.38:0/1596126068 wait complete. 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.116+0000 7fbf131f9640 1 Processor -- start 2024-09-19T05:41:50.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 -- start start 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c106f10 0x7fbf0c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111fc0 con 0x7fbf0c102790 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111e40 con 0x7fbf0c106f10 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf131f9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112140 con 0x7fbf0c101de0 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36762/0 (socket says 172.21.15.38:36762) 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 -- 172.21.15.38:0/733150030 learned_addr learned my addr 172.21.15.38:0/733150030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf11796640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c106f10 0x7fbf0c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf10f95640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 -- 172.21.15.38:0/733150030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c101de0 msgr2=0x7fbf0c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 -- 172.21.15.38:0/733150030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c106f10 msgr2=0x7fbf0c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c106f10 0x7fbf0c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.117+0000 7fbf0bfff640 1 -- 172.21.15.38:0/733150030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c19d490 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d950 tx=0x7fbef400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf09ffb640 1 -- 172.21.15.38:0/733150030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4017030 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf09ffb640 1 -- 172.21.15.38:0/733150030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef4004020 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c19d780 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf09ffb640 1 -- 172.21.15.38:0/733150030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40047a0 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.118+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a5060 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.120+0000 7fbf09ffb640 1 -- 172.21.15.38:0/733150030 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbef40041d0 con 0x7fbf0c102790 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 Processor -- start 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 -- start start 2024-09-19T05:41:50.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a855ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a856e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598a84c3090 con 0x5598a855d1f0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5598a846aa70 con 0x5598a855c8d0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf155d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5598a84bf050 con 0x5598a856bfc0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf09ffb640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 0x7fbef80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36768/0 (socket says 172.21.15.38:36768) 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 -- 172.21.15.38:0/1337494299 learned_addr learned my addr 172.21.15.38:0/1337494299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf11796640 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a855ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf11796640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 0x7fbef80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf09ffb640 1 -- 172.21.15.38:0/733150030 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fbef40cf5b0 con 0x7fbf0c102790 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a856e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 -- 172.21.15.38:0/1337494299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 msgr2=0x5598a856e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a856e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 -- 172.21.15.38:0/1337494299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 msgr2=0x5598a855ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a855ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.121+0000 7fbf10f95640 1 -- 172.21.15.38:0/1337494299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a8477130 con 0x5598a855d1f0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.122+0000 7fbf10f95640 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009890 tx=0x7fbefc02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f302410e65fe407 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.122+0000 7fbf11796640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 0x7fbef80a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf04012870 tx=0x7fbf0400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.122+0000 7fbf02ffd640 1 -- 172.21.15.38:0/1337494299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc003970 con 0x5598a855d1f0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.122+0000 7fbf02ffd640 1 -- 172.21.15.38:0/1337494299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc036030 con 0x5598a855d1f0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.122+0000 7fbf02ffd640 1 -- 172.21.15.38:0/1337494299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0043a0 con 0x5598a855d1f0 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- 172.21.15.38:0/1337494299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 msgr2=0x5598a856ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009890 tx=0x7fbefc02f450 comp rx=0 tx=0).stop 2024-09-19T05:41:50.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- 172.21.15.38:0/1337494299 shutdown_connections 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a856e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a856ba80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 --2- 172.21.15.38:0/1337494299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a855ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- 172.21.15.38:0/1337494299 >> 172.21.15.38:0/1337494299 conn(0x5598a8558f60 msgr2=0x5598a8559340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- 172.21.15.38:0/1337494299 shutdown_connections 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- 172.21.15.38:0/1337494299 wait complete. 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 -- start start 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.123+0000 7fbf155d2740 1 Processor -- start 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855d1f0 0x5598a8568be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a8569120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598a856ea10 con 0x5598a855c8d0 2024-09-19T05:41:50.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5598a847df20 con 0x5598a855d1f0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5598a856eb90 con 0x5598a856bfc0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf155d2740 10 client.0 _ll_register_callbacks cb 0x5598a8447030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36780/0 (socket says 172.21.15.38:36780) 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 -- 172.21.15.38:0/701998507 learned_addr learned my addr 172.21.15.38:0/701998507 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf11796640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a8569120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855d1f0 0x5598a8568be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 -- 172.21.15.38:0/701998507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 msgr2=0x5598a8569120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a8569120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 -- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855d1f0 msgr2=0x5598a8568be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855d1f0 0x5598a8568be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 -- 172.21.15.38:0/701998507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a82342b0 con 0x5598a855c8d0 2024-09-19T05:41:50.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbf10f95640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02aeb0 tx=0x7fbefc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de07d520b056f2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.124+0000 7fbeebfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc04a040 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fbefc004120 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc036030 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbefc05e050 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a856ea10 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc036030 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbefc04d030 con 0x5598a855c8d0 2024-09-19T05:41:50.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 10 client.15411 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 20 client.15411 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a857c640 0x5598a857dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 4 client.15411 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.125+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a84d0600 con 0x5598a857c640 2024-09-19T05:41:50.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.126+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a857c640 0x5598a857dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.126+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a857c640 0x5598a857dab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3617a5e9bce6f98 server_cookie=eaa510b24b45ac46 in_seq=0 out_seq=0 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.126+0000 7fbeebfff640 10 client.15411 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbeebfff640 1 -- 172.21.15.38:0/701998507 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598a84d0600 con 0x5598a857c640 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbeebfff640 10 client.15411 handle_command_reply: tid=0 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 client.15411 shutdown 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 msgr2=0x5598a856b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02aeb0 tx=0x7fbefc003c20 comp rx=0 tx=0).stop 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 shutdown_connections 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a856bfc0 0x5598a8569120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855d1f0 0x5598a8568be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a857c640 0x5598a857dab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 --2- 172.21.15.38:0/701998507 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855c8d0 0x5598a856b890 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbeebfff640 0 client.15411 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:50.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 >> 172.21.15.38:0/701998507 conn(0x5598a8558f60 msgr2=0x5598a855ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 shutdown_connections 2024-09-19T05:41:50.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 1 -- 172.21.15.38:0/701998507 wait complete. 2024-09-19T05:41:50.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.127+0000 7fbf155d2740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T05:41:50.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.145+0000 7fbf155d2740 1 Processor -- start 2024-09-19T05:41:50.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.145+0000 7fbf155d2740 1 -- start start 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 0x5598a857bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a856bfc0 0x5598a8564240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598a857bdf0 con 0x5598a855d1f0 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5598a857c4c0 con 0x5598a856bfc0 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf155d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5598a857cb90 con 0x5598a855c8d0 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36792/0 (socket says 172.21.15.38:36792) 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 -- 172.21.15.38:0/378650463 learned_addr learned my addr 172.21.15.38:0/378650463 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf11796640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a856bfc0 0x5598a8564240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf10f95640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 0x5598a857bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 -- 172.21.15.38:0/378650463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 msgr2=0x5598a857bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 0x5598a857bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 -- 172.21.15.38:0/378650463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a856bfc0 msgr2=0x5598a8564240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a856bfc0 0x5598a8564240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 -- 172.21.15.38:0/378650463 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a856b4b0 con 0x5598a855d1f0 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf10f95640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 0x5598a857bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbef409b510 tx=0x7fbef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74908993a9ddca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.146+0000 7fbf027fc640 1 -- 172.21.15.38:0/378650463 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b9070 con 0x5598a855d1f0 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf027fc640 1 -- 172.21.15.38:0/378650463 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef409bf10 con 0x5598a855d1f0 2024-09-19T05:41:50.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf027fc640 1 -- 172.21.15.38:0/378650463 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b3030 con 0x5598a855d1f0 2024-09-19T05:41:50.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 -- 172.21.15.38:0/378650463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 msgr2=0x5598a857d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbef409b510 tx=0x7fbef40046c0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 -- 172.21.15.38:0/378650463 shutdown_connections 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a856bfc0 0x5598a8564240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a855d1f0 0x5598a857d3e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 --2- 172.21.15.38:0/378650463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855c8d0 0x5598a857bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 -- 172.21.15.38:0/378650463 >> 172.21.15.38:0/378650463 conn(0x5598a8558f60 msgr2=0x5598a8559340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 -- 172.21.15.38:0/378650463 shutdown_connections 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.147+0000 7fbf155d2740 1 -- 172.21.15.38:0/378650463 wait complete. 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 -- start start 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 Processor -- start 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a8580cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855d1f0 0x5598a8581230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5598a857d930 con 0x5598a856bfc0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5598a857d260 con 0x5598a855c8d0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5598a857e000 con 0x5598a855d1f0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf155d2740 10 client.0 _ll_register_callbacks cb 0x7fbf0400bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf11796640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf11796640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36806/0 (socket says 172.21.15.38:36806) 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.148+0000 7fbf11796640 1 -- 172.21.15.38:0/727585040 learned_addr learned my addr 172.21.15.38:0/727585040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf0bfff640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855d1f0 0x5598a8581230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf10f95640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a8580cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 -- 172.21.15.38:0/727585040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855d1f0 msgr2=0x5598a8581230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855d1f0 0x5598a8581230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 -- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 msgr2=0x5598a8580cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a8580cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 -- 172.21.15.38:0/727585040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a85e4f40 con 0x5598a856bfc0 2024-09-19T05:41:50.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf10f95640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a8580cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf11796640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbf040042c0 tx=0x7fbf040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9bb762f1db5fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbeebfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0402f060 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fbf04009dd0 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef409bf10 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbf040035f0 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf155d2740 10 client.15417 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.149+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a85e61f0 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef409bf10 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbf04029070 con 0x5598a856bfc0 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 10 client.15417 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 10 client.15417 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 20 client.15417 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:50.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a85eae50 0x5598a85e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 4 client.15417 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 --> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] -- command(tid 0: {"prefix": "status"}) -- 0x5598a84cab10 con 0x5598a85eae50 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf10f95640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a85eae50 0x5598a85e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbf10f95640 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a85eae50 0x5598a85e9b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33e8cdd822a1c0fb server_cookie=819f76eb91f53314 in_seq=0 out_seq=0 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.150+0000 7fbeebfff640 10 client.15417 ms_handle_connect on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbeebfff640 1 -- 172.21.15.38:0/727585040 <== mds.1 v2:172.21.15.47:6838/1610791508 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5598a84cab10 con 0x5598a85eae50 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbeebfff640 10 client.15417 handle_command_reply: tid=0 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 client.15417 shutdown 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 msgr2=0x5598a8581770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbf040042c0 tx=0x7fbf040046c0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 shutdown_connections 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:6838/1610791508,v1:172.21.15.47:6839/1610791508] conn(0x5598a85eae50 0x5598a85e9b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5598a856bfc0 0x5598a8581770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbeebfff640 0 client.15417 ms_handle_reset on v2:172.21.15.47:6838/1610791508 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598a855d1f0 0x5598a8581230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 --2- 172.21.15.38:0/727585040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598a855c8d0 0x5598a8580cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 >> 172.21.15.38:0/727585040 conn(0x5598a8558f60 msgr2=0x5598a8564240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 shutdown_connections 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 1 -- 172.21.15.38:0/727585040 wait complete. 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.151+0000 7fbf155d2740 20 client.15417 trim_cache size 0 max 16384 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-19T05:41:50.159 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24485, 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 126.078307645, 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 28, 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 178.29612300299999 2024-09-19T05:41:50.160 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 msgr2=0x7fbef80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 0x7fbef80a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf04012870 tx=0x7fbf0400b2e0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 msgr2=0x7fbf0c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d950 tx=0x7fbef400de10 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 shutdown_connections 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fbef80a32c0 0x7fbef80a5780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf0c106f10 0x7fbf0c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf0c102790 0x7fbf0c198a20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 --2- 172.21.15.38:0/733150030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf0c101de0 0x7fbf0c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 >> 172.21.15.38:0/733150030 conn(0x7fbf0c0fd8c0 msgr2=0x7fbf0c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:50.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 shutdown_connections 2024-09-19T05:41:50.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:50.152+0000 7fbf131f9640 1 -- 172.21.15.38:0/733150030 wait complete. 2024-09-19T05:41:50.978 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:41:51.235 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:41:51.621 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:51 smithi038 ceph-mon[31227]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:51 smithi179 ceph-mon[35781]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:51.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:51 smithi047 ceph-mon[35327]: pgmap v371: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 -- 172.21.15.38:0/979881699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648100a80 msgr2=0x7f7648100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 --2- 172.21.15.38:0/979881699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648100a80 0x7f7648100ee0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f763c009930 tx=0x7f763c02a420 comp rx=0 tx=0).stop 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 -- 172.21.15.38:0/979881699 shutdown_connections 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 --2- 172.21.15.38:0/979881699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648101420 0x7f764810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 --2- 172.21.15.38:0/979881699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648100a80 0x7f7648100ee0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 --2- 172.21.15.38:0/979881699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648106a80 0x7f7648106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 -- 172.21.15.38:0/979881699 >> 172.21.15.38:0/979881699 conn(0x7f76480fc7a0 msgr2=0x7f76480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 -- 172.21.15.38:0/979881699 shutdown_connections 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.702+0000 7f764e5df640 1 -- 172.21.15.38:0/979881699 wait complete. 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 Processor -- start 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 -- start start 2024-09-19T05:41:51.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648100a80 0x7f76480fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648101420 0x7f76480fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7648110e80 con 0x7f7648106a80 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7648110d00 con 0x7f7648101420 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764e5df640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7648111000 con 0x7f7648100a80 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764cb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764cb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36828/0 (socket says 172.21.15.38:36828) 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.703+0000 7f764cb7c640 1 -- 172.21.15.38:0/2855849058 learned_addr learned my addr 172.21.15.38:0/2855849058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f76477fe640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648101420 0x7f76480fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f7647fff640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648100a80 0x7f76480fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 -- 172.21.15.38:0/2855849058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648100a80 msgr2=0x7f76480fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648100a80 0x7f76480fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 -- 172.21.15.38:0/2855849058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648101420 msgr2=0x7f76480fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648101420 0x7f76480fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 -- 172.21.15.38:0/2855849058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481a6960 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764cb7c640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f764000eaa0 tx=0x7f764000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f76457fa640 1 -- 172.21.15.38:0/2855849058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764000cdb0 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a6c50 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f76457fa640 1 -- 172.21.15.38:0/2855849058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7640005020 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f76457fa640 1 -- 172.21.15.38:0/2855849058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400116d0 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.704+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a7160 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.706+0000 7f76457fa640 1 -- 172.21.15.38:0/2855849058 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f76400054d0 con 0x7f7648106a80 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 Processor -- start 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 -- start start 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf704670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf704bb0 0x5592bf706fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592bf660090 con 0x5592bf6f99d0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5592bf607a70 con 0x5592bf6fa2f0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76509b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592bf65c050 con 0x5592bf704bb0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36842/0 (socket says 172.21.15.38:36842) 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76477fe640 1 -- 172.21.15.38:0/382326265 learned_addr learned my addr 172.21.15.38:0/382326265 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.707+0000 7f76457fa640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 0x7f76300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f7647fff640 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf704670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf704bb0 0x5592bf706fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f764cb7c640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 0x7f76300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 -- 172.21.15.38:0/382326265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf704bb0 msgr2=0x5592bf706fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf704bb0 0x5592bf706fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76457fa640 1 -- 172.21.15.38:0/2855849058 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f76400cffe0 con 0x7f7648106a80 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 -- 172.21.15.38:0/382326265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 msgr2=0x5592bf704670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf704670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 -- 172.21.15.38:0/382326265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bf614130 con 0x5592bf6f99d0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f76477fe640 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f763c005940 tx=0x7f763c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbca9579932c3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f764cb7c640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 0x7f76300a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7640018000 tx=0x7f7640000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f762e7fc640 1 -- 172.21.15.38:0/382326265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035050 con 0x5592bf6f99d0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f762e7fc640 1 -- 172.21.15.38:0/382326265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c031730 con 0x5592bf6f99d0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.708+0000 7f762e7fc640 1 -- 172.21.15.38:0/382326265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0055b0 con 0x5592bf6f99d0 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- 172.21.15.38:0/382326265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 msgr2=0x5592bf6f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f763c005940 tx=0x7f763c005b50 comp rx=0 tx=0).stop 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- 172.21.15.38:0/382326265 shutdown_connections 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf704bb0 0x5592bf706fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf704670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 --2- 172.21.15.38:0/382326265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf6f9db0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- 172.21.15.38:0/382326265 >> 172.21.15.38:0/382326265 conn(0x5592bf6f5f60 msgr2=0x5592bf6f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- 172.21.15.38:0/382326265 shutdown_connections 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- 172.21.15.38:0/382326265 wait complete. 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.709+0000 7f76509b8740 1 -- start start 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 Processor -- start 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6f99d0 0x5592bf704430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf701780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592bf707780 con 0x5592bf704bb0 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5592bf61af20 con 0x5592bf6fa2f0 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592bf707900 con 0x5592bf6f99d0 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76509b8740 10 client.0 _ll_register_callbacks cb 0x5592bf5e4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:51.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36848/0 (socket says 172.21.15.38:36848) 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 -- 172.21.15.38:0/2298825722 learned_addr learned my addr 172.21.15.38:0/2298825722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6f99d0 0x5592bf704430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f7647fff640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf701780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 -- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6f99d0 msgr2=0x5592bf704430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6f99d0 0x5592bf704430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 -- 172.21.15.38:0/2298825722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 msgr2=0x5592bf701780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf701780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 -- 172.21.15.38:0/2298825722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bf3d12b0 con 0x5592bf704bb0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76477fe640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f763c005c40 tx=0x7f763c005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec6ab550f1baf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.710+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035040 con 0x5592bf704bb0 2024-09-19T05:41:51.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f763c004030 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c031730 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f763c059050 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 10 client.15429 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592bf713090 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c031730 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f763c0035f0 con 0x5592bf704bb0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 10 client.15429 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 10 client.15429 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 20 client.15429 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf7143e0 0x5592bf7165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 4 client.15429 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.711+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592bf66d600 con 0x5592bf7143e0 2024-09-19T05:41:51.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.712+0000 7f7647fff640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf7143e0 0x5592bf7165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.712+0000 7f7647fff640 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf7143e0 0x5592bf7165d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92c57d177c50650 server_cookie=e5f1d10135ddab4a in_seq=0 out_seq=0 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.712+0000 7f76277fe640 10 client.15429 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76277fe640 1 -- 172.21.15.38:0/2298825722 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592bf66d600 con 0x5592bf7143e0 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76277fe640 10 client.15429 handle_command_reply: tid=0 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 client.15429 shutdown 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 msgr2=0x5592bf701cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f763c005c40 tx=0x7f763c005290 comp rx=0 tx=0).stop 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 shutdown_connections 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf7143e0 0x5592bf7165d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf704bb0 0x5592bf701cc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76277fe640 0 client.15429 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf6fa2f0 0x5592bf701780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 --2- 172.21.15.38:0/2298825722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6f99d0 0x5592bf704430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 >> 172.21.15.38:0/2298825722 conn(0x5592bf6f5f60 msgr2=0x5592bf6fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 shutdown_connections 2024-09-19T05:41:51.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 1 -- 172.21.15.38:0/2298825722 wait complete. 2024-09-19T05:41:51.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.713+0000 7f76509b8740 20 client.15429 trim_cache size 0 max 16384 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 Processor -- start 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 -- start start 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf715830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf700050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592bf714910 con 0x5592bf6f99d0 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5592bf714fe0 con 0x5592bf704bb0 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76509b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592bf7156b0 con 0x5592bf6fa2f0 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36852/0 (socket says 172.21.15.38:36852) 2024-09-19T05:41:51.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 -- 172.21.15.38:0/3931675862 learned_addr learned my addr 172.21.15.38:0/3931675862 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f7647fff640 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf715830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 -- 172.21.15.38:0/3931675862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 msgr2=0x5592bf715830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf715830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 -- 172.21.15.38:0/3931675862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 msgr2=0x5592bf700050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf700050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 -- 172.21.15.38:0/3931675862 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bf6f59e0 con 0x5592bf6f99d0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f7647fff640 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf715830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.732+0000 7f76477fe640 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f763c02aae0 tx=0x7f763c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65930c551bd2c001 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f762dffb640 1 -- 172.21.15.38:0/3931675862 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c062070 con 0x5592bf6f99d0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f762dffb640 1 -- 172.21.15.38:0/3931675862 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c050070 con 0x5592bf6f99d0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f762dffb640 1 -- 172.21.15.38:0/3931675862 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c035040 con 0x5592bf6f99d0 2024-09-19T05:41:51.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 -- 172.21.15.38:0/3931675862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 msgr2=0x5592bf714a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f763c02aae0 tx=0x7f763c005e30 comp rx=0 tx=0).stop 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 -- 172.21.15.38:0/3931675862 shutdown_connections 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf700050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf715830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 --2- 172.21.15.38:0/3931675862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf714a90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 -- 172.21.15.38:0/3931675862 >> 172.21.15.38:0/3931675862 conn(0x5592bf6f5f60 msgr2=0x5592bf6f0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 -- 172.21.15.38:0/3931675862 shutdown_connections 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.733+0000 7f76509b8740 1 -- 172.21.15.38:0/3931675862 wait complete. 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 -- start start 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 Processor -- start 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf71f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf717fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5592bf716450 con 0x5592bf6f99d0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5592bf715d80 con 0x5592bf704bb0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5592bf716b20 con 0x5592bf6fa2f0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.734+0000 7f76509b8740 10 client.0 _ll_register_callbacks cb 0x7f763c032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36864/0 (socket says 172.21.15.38:36864) 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 -- 172.21.15.38:0/2493843666 learned_addr learned my addr 172.21.15.38:0/2493843666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f7647fff640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf71f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf717fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 -- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 msgr2=0x5592bf71f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf71f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 -- 172.21.15.38:0/2493843666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 msgr2=0x5592bf717fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf717fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 -- 172.21.15.38:0/2493843666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bf718830 con 0x5592bf6f99d0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76477fe640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f763c060040 tx=0x7f763c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aec290921476c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76277fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c062060 con 0x5592bf6f99d0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c050070 con 0x5592bf6f99d0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f763c003d50 con 0x5592bf6f99d0 2024-09-19T05:41:51.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.735+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f763c035040 con 0x5592bf6f99d0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 10 client.15435 fetch_fsmap learned FSMap version 28 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592bf71d160 con 0x5592bf6f99d0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c050070 con 0x5592bf6f99d0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f763c04b9b0 con 0x5592bf6f99d0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 10 client.15435 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 10 client.15435 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 20 client.15435 populate_metadata read hostname 'smithi038' 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf783ab0 0x5592bf7827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 4 client.15435 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "status"}) -- 0x5592bf667b10 con 0x5592bf783ab0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.736+0000 7f7647fff640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf783ab0 0x5592bf7827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f7647fff640 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf783ab0 0x5592bf7827c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be64969c02282d94 server_cookie=2bcf5f8347d2551a in_seq=0 out_seq=0 2024-09-19T05:41:51.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76277fe640 10 client.15435 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76277fe640 1 -- 172.21.15.38:0/2493843666 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5592bf667b10 con 0x5592bf783ab0 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76277fe640 10 client.15435 handle_command_reply: tid=0 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 client.15435 shutdown 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 msgr2=0x5592bf71eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f763c060040 tx=0x7f763c0315b0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 shutdown_connections 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5592bf783ab0 0x5592bf7827c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bf704bb0 0x5592bf717fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76277fe640 0 client.15435 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5592bf6fa2f0 0x5592bf71f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 --2- 172.21.15.38:0/2493843666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5592bf6f99d0 0x5592bf71eb10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.737+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 >> 172.21.15.38:0/2493843666 conn(0x5592bf6f5f60 msgr2=0x5592bf6f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 shutdown_connections 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f76509b8740 1 -- 172.21.15.38:0/2493843666 wait complete. 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f76509b8740 20 client.15435 trim_cache size 0 max 16384 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "44ff0e58-7648-11ef-bceb-c7b262605968", 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-19T05:41:51.751 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24496, 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 168.836685655, 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 28, 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 91, 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 188.69543618099999 2024-09-19T05:41:51.752 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 msgr2=0x7f76300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 0x7f76300a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7640018000 tx=0x7f7640000f40 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 msgr2=0x7f76480fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f764000eaa0 tx=0x7f764000ef60 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 shutdown_connections 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f76300a3240 0x7f76300a5700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7648106a80 0x7f76480fd7c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7648101420 0x7f76480fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 --2- 172.21.15.38:0/2855849058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7648100a80 0x7f76480fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:41:51.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 >> 172.21.15.38:0/2855849058 conn(0x7f76480fc7a0 msgr2=0x7f764806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:41:51.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 shutdown_connections 2024-09-19T05:41:51.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:41:51.738+0000 7f764e5df640 1 -- 172.21.15.38:0/2855849058 wait complete. 2024-09-19T05:41:52.556 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T05:41:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:53 smithi038 ceph-mon[31227]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T05:41:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:53 smithi179 ceph-mon[35781]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T05:41:53.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:53 smithi047 ceph-mon[35327]: pgmap v372: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-19T05:41:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:55 smithi038 ceph-mon[31227]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:55 smithi179 ceph-mon[35781]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:55 smithi047 ceph-mon[35327]: pgmap v373: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:41:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:57 smithi038 ceph-mon[31227]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:41:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:57 smithi179 ceph-mon[35781]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:41:57.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:57 smithi047 ceph-mon[35327]: pgmap v374: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:41:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:41:59 smithi038 ceph-mon[31227]: pgmap v375: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:41:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:41:59 smithi179 ceph-mon[35781]: pgmap v375: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:41:59.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:41:59 smithi047 ceph-mon[35327]: pgmap v375: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:42:01.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:01 smithi038 ceph-mon[31227]: pgmap v376: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:01 smithi179 ceph-mon[35781]: pgmap v376: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:01.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:01 smithi047 ceph-mon[35327]: pgmap v376: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:03 smithi179 ceph-mon[35781]: pgmap v377: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:03 smithi047 ceph-mon[35327]: pgmap v377: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:03.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:03 smithi038 ceph-mon[31227]: pgmap v377: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:04.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:04 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:42:04.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:04 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:42:04.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:04 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:42:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: pgmap v378: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:42:05.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:42:05.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: pgmap v378: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:42:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:42:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: pgmap v378: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:42:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:42:07.557 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T05:42:07.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:42:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:07 smithi179 ceph-mon[35781]: pgmap v379: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:07 smithi047 ceph-mon[35327]: pgmap v379: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:07.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:07 smithi038 ceph-mon[31227]: pgmap v379: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 -- 172.21.15.38:0/2544920745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60103740 msgr2=0x7f7e60107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 --2- 172.21.15.38:0/2544920745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60103740 0x7f7e60107b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009930 tx=0x7f7e5402a420 comp rx=0 tx=0).stop 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 -- 172.21.15.38:0/2544920745 shutdown_connections 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 --2- 172.21.15.38:0/2544920745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60108290 0x7f7e6010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 --2- 172.21.15.38:0/2544920745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60103740 0x7f7e60107b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 --2- 172.21.15.38:0/2544920745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60102d90 0x7f7e60103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 -- 172.21.15.38:0/2544920745 >> 172.21.15.38:0/2544920745 conn(0x7f7e600fc7d0 msgr2=0x7f7e600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 -- 172.21.15.38:0/2544920745 shutdown_connections 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.318+0000 7f7e654ec640 1 -- 172.21.15.38:0/2544920745 wait complete. 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 Processor -- start 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 -- start start 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 0x7f7e60198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 0x7f7e6019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60112090 con 0x7f7e60102d90 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60111f10 con 0x7f7e60103740 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e654ec640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7e60112210 con 0x7f7e60108290 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.319+0000 7f7e5effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:08.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37500/0 (socket says 172.21.15.38:37500) 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 -- 172.21.15.38:0/1998224468 learned_addr learned my addr 172.21.15.38:0/1998224468 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5e7fc640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 0x7f7e60198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5f7fe640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 0x7f7e6019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 -- 172.21.15.38:0/1998224468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 msgr2=0x7f7e6019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 0x7f7e6019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 -- 172.21.15.38:0/1998224468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 msgr2=0x7f7e60198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 0x7f7e60198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 -- 172.21.15.38:0/1998224468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6019d570 con 0x7f7e60102d90 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5f7fe640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 0x7f7e6019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5e7fc640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 0x7f7e60198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e5effd640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9f0 tx=0x7f7e5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cd80 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50010910 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6019d860 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50018990 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.320+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601a5140 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.321+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60103170 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.324+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7e50010ac0 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.325+0000 7f7e47fff640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 0x7f7e480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.325+0000 7f7e5e7fc640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 0x7f7e480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.325+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7e500d0580 con 0x7f7e60102d90 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.325+0000 7f7e5e7fc640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 0x7f7e480a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e54006010 tx=0x7f7e54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:08.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.328+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e5009cee0 con 0x7f7e60102d90 2024-09-19T05:42:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.558+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e6010a2a0 con 0x7f7e60102d90 2024-09-19T05:42:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.559+0000 7f7e47fff640 1 -- 172.21.15.38:0/1998224468 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f7e5009c7e0 con 0x7f7e60102d90 2024-09-19T05:42:08.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:42:08.562 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 msgr2=0x7f7e480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 0x7f7e480a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e54006010 tx=0x7f7e54005a20 comp rx=0 tx=0).stop 2024-09-19T05:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 msgr2=0x7f7e601985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000e9f0 tx=0x7f7e5000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 shutdown_connections 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e480a3240 0x7f7e480a5700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e60108290 0x7f7e6019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e60103740 0x7f7e60198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 --2- 172.21.15.38:0/1998224468 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e60102d90 0x7f7e601985c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.561+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 >> 172.21.15.38:0/1998224468 conn(0x7f7e600fc7d0 msgr2=0x7f7e60105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.562+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 shutdown_connections 2024-09-19T05:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:08.562+0000 7f7e654ec640 1 -- 172.21.15.38:0/1998224468 wait complete. 2024-09-19T05:42:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:09 smithi179 ceph-mon[35781]: pgmap v380: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:42:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:09 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1998224468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:09 smithi047 ceph-mon[35327]: pgmap v380: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:42:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:09 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1998224468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:09.795 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:09 smithi038 ceph-mon[31227]: pgmap v380: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:42:09.795 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:09 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1998224468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:11 smithi179 ceph-mon[35781]: pgmap v381: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:11 smithi047 ceph-mon[35327]: pgmap v381: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:11.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:11 smithi038 ceph-mon[31227]: pgmap v381: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:13 smithi179 ceph-mon[35781]: pgmap v382: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:13 smithi047 ceph-mon[35327]: pgmap v382: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:13.753 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:13 smithi038 ceph-mon[31227]: pgmap v382: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:13.782 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T05:42:13.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T05:42:14.100 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 -- 172.21.15.38:0/3751627620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e201036e0 msgr2=0x7f5e20107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 --2- 172.21.15.38:0/3751627620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e201036e0 0x7f5e20107b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009930 tx=0x7f5e1402a420 comp rx=0 tx=0).stop 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 -- 172.21.15.38:0/3751627620 shutdown_connections 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 --2- 172.21.15.38:0/3751627620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 0x7f5e2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 --2- 172.21.15.38:0/3751627620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e201036e0 0x7f5e20107b90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 --2- 172.21.15.38:0/3751627620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e20102d30 0x7f5e20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 -- 172.21.15.38:0/3751627620 >> 172.21.15.38:0/3751627620 conn(0x7f5e200fc7d0 msgr2=0x7f5e200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 -- 172.21.15.38:0/3751627620 shutdown_connections 2024-09-19T05:42:14.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.654+0000 7f5e26852640 1 -- 172.21.15.38:0/3751627620 wait complete. 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 Processor -- start 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 -- start start 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 0x7f5e20198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 0x7f5e2019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e201120a0 con 0x7f5e20102d30 2024-09-19T05:42:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e20111f20 con 0x7f5e201036e0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.655+0000 7f5e26852640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e20112220 con 0x7f5e201082a0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37506/0 (socket says 172.21.15.38:37506) 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 -- 172.21.15.38:0/2693432279 learned_addr learned my addr 172.21.15.38:0/2693432279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 0x7f5e20198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e24def640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 0x7f5e2019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 -- 172.21.15.38:0/2693432279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 msgr2=0x7f5e2019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 0x7f5e2019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 -- 172.21.15.38:0/2693432279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 msgr2=0x7f5e20198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 0x7f5e20198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 -- 172.21.15.38:0/2693432279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2019d5b0 con 0x7f5e20102d30 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 0x7f5e20198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000e9f0 tx=0x7f5e1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.656+0000 7f5e1d7fa640 1 -- 172.21.15.38:0/2693432279 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000cd80 con 0x7f5e20102d30 2024-09-19T05:42:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.657+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2019d8a0 con 0x7f5e20102d30 2024-09-19T05:42:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.657+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e201a5180 con 0x7f5e20102d30 2024-09-19T05:42:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.657+0000 7f5e1d7fa640 1 -- 172.21.15.38:0/2693432279 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e10010910 con 0x7f5e20102d30 2024-09-19T05:42:14.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.657+0000 7f5e1d7fa640 1 -- 172.21.15.38:0/2693432279 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100188f0 con 0x7f5e20102d30 2024-09-19T05:42:14.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.660+0000 7f5e28c2b740 1 Processor -- start 2024-09-19T05:42:14.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.660+0000 7f5e28c2b740 1 -- start start 2024-09-19T05:42:14.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.660+0000 7f5e1d7fa640 1 -- 172.21.15.38:0/2693432279 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5e10018bd0 con 0x7f5e20102d30 2024-09-19T05:42:14.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a3fd0 0x5564b49a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5564b49052a0 con 0x5564b49a50a0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5564b48acb10 con 0x5564b49a3fd0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5564b4901260 con 0x5564b49b2fe0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e24def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e24def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37508/0 (socket says 172.21.15.38:37508) 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e24def640 1 -- 172.21.15.38:0/2149973941 learned_addr learned my addr 172.21.15.38:0/2149973941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a3fd0 0x5564b49a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 -- 172.21.15.38:0/2149973941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 msgr2=0x5564b49b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e1d7fa640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 0x7f5e080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.661+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 -- 172.21.15.38:0/2149973941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a3fd0 msgr2=0x5564b49a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a3fd0 0x5564b49a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 -- 172.21.15.38:0/2149973941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b4676b90 con 0x5564b49a50a0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e1d7fa640 1 -- 172.21.15.38:0/2693432279 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5e100d00d0 con 0x7f5e20102d30 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 0x7f5e080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.662+0000 7f5e24def640 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e18008d40 tx=0x7f5e1800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41186c105087caf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.663+0000 7f5e067fc640 1 -- 172.21.15.38:0/2149973941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18020070 con 0x5564b49a50a0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.663+0000 7f5e067fc640 1 -- 172.21.15.38:0/2149973941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e18010e20 con 0x5564b49a50a0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.663+0000 7f5e067fc640 1 -- 172.21.15.38:0/2149973941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18014070 con 0x5564b49a50a0 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2149973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 msgr2=0x5564b49a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e24def640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 0x7f5e080a5780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e1801d000 tx=0x7f5e18020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e18008d40 tx=0x7f5e1800e720 comp rx=0 tx=0).stop 2024-09-19T05:42:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2149973941 shutdown_connections 2024-09-19T05:42:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a3fd0 0x5564b49a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2149973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49a3a90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2149973941 >> 172.21.15.38:0/2149973941 conn(0x5564b49a0100 msgr2=0x5564b49a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2149973941 shutdown_connections 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2149973941 wait complete. 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.664+0000 7f5e28c2b740 1 -- start start 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 Processor -- start 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 0x5564b49b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e24def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 0x5564b49b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:44696/0 (socket says 172.21.15.38:44696) 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e1ffff640 1 -- 172.21.15.38:0/1909155300 learned_addr learned my addr 172.21.15.38:0/1909155300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5564b49b5dc0 con 0x5564b49b2fe0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5564b488d6e0 con 0x5564b49a50a0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5564b49b5f40 con 0x5564b49a3fd0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.665+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 -- 172.21.15.38:0/1909155300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 msgr2=0x5564b49ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 -- 172.21.15.38:0/1909155300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 msgr2=0x5564b49b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 0x5564b49b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e28c2b740 10 client.0 _ll_register_callbacks cb 0x5564b48b1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b49b28c0 con 0x5564b49a3fd0 2024-09-19T05:42:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e24def640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 0x5564b49b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e18010e20 con 0x5564b49a3fd0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e10098ca0 tx=0x7f5e1009be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa90e92315342045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5dff7fe640 10 client.25201 ms_handle_connect on v2:172.21.15.179:3300/0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5dff7fe640 1 -- 172.21.15.38:0/1909155300 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100d2310 con 0x5564b49a3fd0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5dff7fe640 1 -- 172.21.15.38:0/1909155300 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5e100bc0b0 con 0x5564b49a3fd0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.666+0000 7f5dff7fe640 1 -- 172.21.15.38:0/1909155300 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e100bc2d0 con 0x5564b49a3fd0 2024-09-19T05:42:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 10 client.25201 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5564b49ca430 con 0x5564b49a3fd0 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5dff7fe640 1 -- 172.21.15.38:0/1909155300 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5e100a0150 con 0x5564b49a3fd0 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 10 client.25201 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 10 client.25201 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 20 client.25201 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b49cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 4 client.25201 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:42:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564b49129d0 con 0x5564b49cb780 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.667+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b49cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.668+0000 7f5dff7fe640 10 client.25201 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.668+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b49cdb70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498197820ffea393 server_cookie=5f6b0c27dc2b6409 in_seq=0 out_seq=0 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.668+0000 7f5dff7fe640 1 -- 172.21.15.38:0/1909155300 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564b49129d0 con 0x5564b49cb780 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.668+0000 7f5dff7fe640 10 client.25201 handle_command_reply: tid=0 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 client.25201 shutdown 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 msgr2=0x5564b49ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e10098ca0 tx=0x7f5e1009be10 comp rx=0 tx=0).stop 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 shutdown_connections 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b49cdb70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49b2fe0 0x5564b49b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5dff7fe640 0 client.25201 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/1909155300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49ac360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 >> 172.21.15.38:0/1909155300 conn(0x5564b49a0100 msgr2=0x5564b49a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 shutdown_connections 2024-09-19T05:42:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 1 -- 172.21.15.38:0/1909155300 wait complete. 2024-09-19T05:42:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.669+0000 7f5e28c2b740 20 client.25201 trim_cache size 0 max 16384 2024-09-19T05:42:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 Processor -- start 2024-09-19T05:42:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 -- start start 2024-09-19T05:42:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49abf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c00fc00 con 0x5564b49a3fd0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c011740 con 0x5564b49a50a0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0102d0 con 0x5564b49b2fe0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e1ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37528/0 (socket says 172.21.15.38:37528) 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e1ffff640 1 -- 172.21.15.38:0/665804449 learned_addr learned my addr 172.21.15.38:0/665804449 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e24def640 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.704+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 -- 172.21.15.38:0/665804449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 msgr2=0x5564b49abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 -- 172.21.15.38:0/665804449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 msgr2=0x5564b49b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 -- 172.21.15.38:0/665804449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b499ebf0 con 0x5564b49a3fd0 2024-09-19T05:42:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e1009bf60 tx=0x7f5e10086000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4dfa25618cd228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e05ffb640 1 -- 172.21.15.38:0/665804449 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100a0680 con 0x5564b49a3fd0 2024-09-19T05:42:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e05ffb640 1 -- 172.21.15.38:0/665804449 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e18010e20 con 0x5564b49a3fd0 2024-09-19T05:42:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.705+0000 7f5e05ffb640 1 -- 172.21.15.38:0/665804449 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100d2370 con 0x5564b49a3fd0 2024-09-19T05:42:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 -- 172.21.15.38:0/665804449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 msgr2=0x5564b49b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e1009bf60 tx=0x7f5e10086000 comp rx=0 tx=0).stop 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 -- 172.21.15.38:0/665804449 shutdown_connections 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49b2fe0 0x5564b49abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49a50a0 0x5564b49b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/665804449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a3fd0 0x5564b49b1040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 -- 172.21.15.38:0/665804449 >> 172.21.15.38:0/665804449 conn(0x5564b49a0100 msgr2=0x5564b49a6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 -- 172.21.15.38:0/665804449 shutdown_connections 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.706+0000 7f5e28c2b740 1 -- 172.21.15.38:0/665804449 wait complete. 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 -- start start 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 Processor -- start 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 0x5564b49dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c011070 con 0x5564b49a50a0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0109a0 con 0x5564b49b2fe0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c00dbe0 con 0x5564b49a3fd0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e28c2b740 10 client.0 _ll_register_callbacks cb 0x7f5e10098570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37538/0 (socket says 172.21.15.38:37538) 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.707+0000 7f5e1f7fe640 1 -- 172.21.15.38:0/2481538431 learned_addr learned my addr 172.21.15.38:0/2481538431 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e24def640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 0x5564b49dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 -- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 msgr2=0x5564b49dbf60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49dbf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 -- 172.21.15.38:0/2481538431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 msgr2=0x5564b49dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 0x5564b49dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 -- 172.21.15.38:0/2481538431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b4a36340 con 0x5564b49a50a0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e24def640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 0x5564b49dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e1f7fe640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005ab0 tx=0x7f5e140042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f69c7f97602a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5dff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14035040 con 0x5564b49a50a0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5e14005450 con 0x5564b49a50a0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.708+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e18010e20 con 0x5564b49a50a0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.709+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e14052050 con 0x5564b49a50a0 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.709+0000 7f5e28c2b740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.709+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564b4a37640 con 0x5564b49a50a0 2024-09-19T05:42:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.709+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e14004ba0 con 0x5564b49a50a0 2024-09-19T05:42:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.709+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5e140039d0 con 0x5564b49a50a0 2024-09-19T05:42:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 10 client.15459 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 20 client.15459 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b4a39ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 4 client.15459 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x7f5e18010e20 con 0x5564b49cb780 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.710+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b4a39ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.711+0000 7f5e1ffff640 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b4a39ec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910dd4e401ff28 server_cookie=c808bc98cb0beb41 in_seq=0 out_seq=0 2024-09-19T05:42:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.711+0000 7f5dff7fe640 10 client.15459 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5dff7fe640 1 -- 172.21.15.38:0/2481538431 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f5e18010e20 con 0x5564b49cb780 2024-09-19T05:42:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5dff7fe640 10 client.15459 handle_command_reply: tid=0 2024-09-19T05:42:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 client.15459 shutdown 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 msgr2=0x5564b49dc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005ab0 tx=0x7f5e140042b0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 shutdown_connections 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5564b49cb780 0x5564b4a39ec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564b49b2fe0 0x5564b49dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5564b49a50a0 0x5564b49dc4a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 --2- 172.21.15.38:0/2481538431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5564b49a3fd0 0x5564b49dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 >> 172.21.15.38:0/2481538431 conn(0x5564b49a0100 msgr2=0x5564b49b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.722+0000 7f5dff7fe640 0 client.15459 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 shutdown_connections 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e28c2b740 1 -- 172.21.15.38:0/2481538431 wait complete. 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e28c2b740 20 client.15459 trim_cache size 0 max 16384 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:42:14.726 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 2, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "bf2a5356": { 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 2, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 131.2, 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:42:14.727 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:14.728 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 msgr2=0x7f5e080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 0x7f5e080a5780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e1801d000 tx=0x7f5e18020000 comp rx=0 tx=0).stop 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 msgr2=0x7f5e20198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.723+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000e9f0 tx=0x7f5e1000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 shutdown_connections 2024-09-19T05:42:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5e080a32c0 0x7f5e080a5780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e201082a0 0x7f5e2019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e201036e0 0x7f5e20198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 --2- 172.21.15.38:0/2693432279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e20102d30 0x7f5e20198600 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 >> 172.21.15.38:0/2693432279 conn(0x7f5e200fc7d0 msgr2=0x7f5e20105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 shutdown_connections 2024-09-19T05:42:14.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:14.724+0000 7f5e26852640 1 -- 172.21.15.38:0/2693432279 wait complete. 2024-09-19T05:42:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:15 smithi038 ceph-mon[31227]: pgmap v383: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:15.698 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bf2a5356', quiesce took 8.1 seconds. Will release after: 3.0999999999999996 2024-09-19T05:42:15.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:15 smithi179 ceph-mon[35781]: pgmap v383: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:15 smithi047 ceph-mon[35327]: pgmap v383: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:17 smithi047 ceph-mon[35327]: pgmap v384: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:17 smithi038 ceph-mon[31227]: pgmap v384: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:17 smithi179 ceph-mon[35781]: pgmap v384: 129 pgs: 129 active+clean; 659 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:18.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:42:19.178 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:19.627 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:19 smithi038 ceph-mon[31227]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:19.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:19 smithi047 ceph-mon[35327]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.751+0000 7f759d001640 1 -- 172.21.15.38:0/1135876401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7598101290 msgr2=0x7f759810d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.751+0000 7f759d001640 1 --2- 172.21.15.38:0/1135876401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7598101290 0x7f759810d7c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7590009cd0 tx=0x7f759002a8e0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 -- 172.21.15.38:0/1135876401 shutdown_connections 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 --2- 172.21.15.38:0/1135876401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7598101290 0x7f759810d7c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 --2- 172.21.15.38:0/1135876401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981008f0 0x7f7598100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 --2- 172.21.15.38:0/1135876401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75981068f0 0x7f7598106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 -- 172.21.15.38:0/1135876401 >> 172.21.15.38:0/1135876401 conn(0x7f75980fc630 msgr2=0x7f75980fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 -- 172.21.15.38:0/1135876401 shutdown_connections 2024-09-19T05:42:19.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.752+0000 7f759d001640 1 -- 172.21.15.38:0/1135876401 wait complete. 2024-09-19T05:42:19.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.753+0000 7f759d001640 1 Processor -- start 2024-09-19T05:42:19.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.753+0000 7f759d001640 1 -- start start 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.753+0000 7f759d001640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.753+0000 7f759d001640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 0x7f7598199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759d001640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981068f0 0x7f75981a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759d001640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7598110d40 con 0x7f75981008f0 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759d001640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7598110bc0 con 0x7f75981068f0 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759d001640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7598110ec0 con 0x7f7598101290 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f7596d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 0x7f7598199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759759e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981068f0 0x7f75981a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f7596d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49342/0 (socket says 172.21.15.38:49342) 2024-09-19T05:42:19.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f7596d9d640 1 -- 172.21.15.38:0/848871395 learned_addr learned my addr 172.21.15.38:0/848871395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 -- 172.21.15.38:0/848871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981068f0 msgr2=0x7f75981a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981068f0 0x7f75981a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 -- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 msgr2=0x7f7598199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f759659c640 1 -- 172.21.15.38:0/848871395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75981a46f0 con 0x7f7598101290 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.754+0000 7f7596d9d640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f759659c640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 0x7f7598199670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9f0 tx=0x7f758c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:19.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00ccc0 con 0x7f7598101290 2024-09-19T05:42:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c010910 con 0x7f7598101290 2024-09-19T05:42:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c018960 con 0x7f7598101290 2024-09-19T05:42:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75981a48c0 con 0x7f7598101290 2024-09-19T05:42:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.755+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981a4cc0 con 0x7f7598101290 2024-09-19T05:42:19.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.757+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7598100d50 con 0x7f7598101290 2024-09-19T05:42:19.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.759+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f758c010ac0 con 0x7f7598101290 2024-09-19T05:42:19.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.760+0000 7f757ffff640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 0x7f75800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:19.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.760+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f758c0d0170 con 0x7f7598101290 2024-09-19T05:42:19.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.760+0000 7f7596d9d640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 0x7f75800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:19.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.761+0000 7f7596d9d640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 0x7f75800a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7588009cb0 tx=0x7f7588005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:19.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:19.764+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f758c09ca70 con 0x7f7598101290 2024-09-19T05:42:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:19 smithi179 ceph-mon[35781]: pgmap v385: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:20.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.198+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75981a4f80 con 0x7f7598101290 2024-09-19T05:42:20.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.199+0000 7f757ffff640 1 -- 172.21.15.38:0/848871395 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f758c09c370 con 0x7f7598101290 2024-09-19T05:42:20.201 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:42:20.201 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:42:20.202 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 msgr2=0x7f75800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 0x7f75800a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7588009cb0 tx=0x7f7588005f90 comp rx=0 tx=0).stop 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 msgr2=0x7f7598199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 0x7f7598199670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e9f0 tx=0x7f758c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 shutdown_connections 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75800a3240 0x7f75800a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75981068f0 0x7f75981a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7598101290 0x7f7598199670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 --2- 172.21.15.38:0/848871395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75981008f0 0x7f7598199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 >> 172.21.15.38:0/848871395 conn(0x7f75980fc630 msgr2=0x7f75980fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.202+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 shutdown_connections 2024-09-19T05:42:20.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:20.203+0000 7f759d001640 1 -- 172.21.15.38:0/848871395 wait complete. 2024-09-19T05:42:20.458 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:20 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/848871395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:20.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:20 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/848871395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:20.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:20 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/848871395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:21.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id bf2a5356 --release --await' 2024-09-19T05:42:21.035 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id bf2a5356 --release --await 2024-09-19T05:42:21.311 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:21 smithi047 ceph-mon[35327]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:21.770 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:21 smithi038 ceph-mon[31227]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/1447726514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a41036e0 msgr2=0x7fa4a410bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/1447726514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a41036e0 0x7fa4a410bbd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa494009900 tx=0x7fa49402a480 comp rx=0 tx=0).stop 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/1447726514 shutdown_connections 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/1447726514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a410c1a0 0x7fa4a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/1447726514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a41036e0 0x7fa4a410bbd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/1447726514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a4102dc0 0x7fa4a41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/1447726514 >> 172.21.15.38:0/1447726514 conn(0x7fa4a40fc7a0 msgr2=0x7fa4a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/1447726514 shutdown_connections 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.863+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/1447726514 wait complete. 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 Processor -- start 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 -- start start 2024-09-19T05:42:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 0x7fa4a4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a41036e0 0x7fa4a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111f70 con 0x7fa4a410c1a0 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111df0 con 0x7fa4a4102dc0 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.864+0000 7fa4ab2b4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41120f0 con 0x7fa4a41036e0 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49370/0 (socket says 172.21.15.38:49370) 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 -- 172.21.15.38:0/3184262497 learned_addr learned my addr 172.21.15.38:0/3184262497 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 -- 172.21.15.38:0/3184262497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a41036e0 msgr2=0x7fa4a4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a41036e0 0x7fa4a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.865+0000 7fa4a9851640 1 -- 172.21.15.38:0/3184262497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 msgr2=0x7fa4a4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 0x7fa4a4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4a9851640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 0x7fa4a4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4a9851640 1 -- 172.21.15.38:0/3184262497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a419d3e0 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 0x7fa4a4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4a9851640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00eb10 tx=0x7fa49c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa49a7fc640 1 -- 172.21.15.38:0/3184262497 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004b20 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa49a7fc640 1 -- 172.21.15.38:0/3184262497 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49c004cc0 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa49a7fc640 1 -- 172.21.15.38:0/3184262497 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c010760 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a419d6d0 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.866+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a4fb0 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.869+0000 7fa4ad68d740 1 Processor -- start 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.870+0000 7fa49a7fc640 1 -- 172.21.15.38:0/3184262497 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa49c005020 con 0x7fa4a410c1a0 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.870+0000 7fa4ad68d740 1 -- start start 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.870+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a35840 0x55ab02a44110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49380/0 (socket says 172.21.15.38:49380) 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 -- 172.21.15.38:0/3178998466 learned_addr learned my addr 172.21.15.38:0/3178998466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a44650 0x55ab02a46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab0299a620 con 0x55ab02a34f20 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ab02941e50 con 0x55ab02a44650 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ab029965e0 con 0x55ab02a35840 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa49a7fc640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 0x7fa4900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a35840 0x55ab02a44110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a44650 0x55ab02a46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 -- 172.21.15.38:0/3178998466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a35840 msgr2=0x55ab02a44110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a35840 0x55ab02a44110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 -- 172.21.15.38:0/3178998466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a44650 msgr2=0x55ab02a46a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a44650 0x55ab02a46a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 -- 172.21.15.38:0/3178998466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab02932d40 con 0x55ab02a34f20 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa49a7fc640 1 -- 172.21.15.38:0/3184262497 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fa49c0d0320 con 0x7fa4a410c1a0 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a9851640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 0x7fa4900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.871+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa494009a20 tx=0x7fa494031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77c7872bba17a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.872+0000 7fa48f7fe640 1 -- 172.21.15.38:0/3178998466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494035040 con 0x55ab02a34f20 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.872+0000 7fa48f7fe640 1 -- 172.21.15.38:0/3178998466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494042070 con 0x55ab02a34f20 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.872+0000 7fa48f7fe640 1 -- 172.21.15.38:0/3178998466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494005450 con 0x55ab02a34f20 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4a9851640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 0x7fa4900a5780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0989e0 tx=0x7fa49c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 msgr2=0x55ab02a35300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa494009a20 tx=0x7fa494031400 comp rx=0 tx=0).stop 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 shutdown_connections 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a44650 0x55ab02a46a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a35840 0x55ab02a44110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3178998466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a35300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 >> 172.21.15.38:0/3178998466 conn(0x55ab02a315f0 msgr2=0x55ab02a319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 shutdown_connections 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3178998466 wait complete. 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 -- start start 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.873+0000 7fa4ad68d740 1 Processor -- start 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a3dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a44650 0x55ab02a420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ab02a37e80 con 0x55ab02a44650 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ab02922f10 con 0x55ab02a35840 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ab02a38000 con 0x55ab02a34f20 2024-09-19T05:42:21.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4a9050640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a3dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:41302/0 (socket says 172.21.15.38:41302) 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4a884f640 1 -- 172.21.15.38:0/1820610520 learned_addr learned my addr 172.21.15.38:0/1820610520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.874+0000 7fa4ad68d740 10 client.0 _ll_register_callbacks cb 0x55ab02946db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 -- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 msgr2=0x55ab02a3d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 -- 172.21.15.38:0/1820610520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a44650 msgr2=0x55ab02a420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a44650 0x55ab02a420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 -- 172.21.15.38:0/1820610520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab026a9580 con 0x55ab02a35840 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a884f640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4a9050640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a3dd70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009f30 tx=0x7fa4a0011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd03dec09e1efc75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00129e0 con 0x55ab02a35840 2024-09-19T05:42:21.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494042070 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.875+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fa4a0012b80 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.876+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4a0012d80 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.876+0000 7fa4ad68d740 10 client.25205 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.876+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab02773960 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.876+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494042070 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.876+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa4a000f040 con 0x55ab02a35840 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 10 client.25205 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 10 client.25205 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 20 client.25205 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02a522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 4 client.25205 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab029a7b90 con 0x55ab02a4fe80 2024-09-19T05:42:21.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4a884f640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02a522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa4a884f640 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02a522d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5488bcffe83eff4f server_cookie=e9ddcb8e194a4312 in_seq=0 out_seq=0 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.877+0000 7fa48cff9640 10 client.25205 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.879+0000 7fa48cff9640 1 -- 172.21.15.38:0/1820610520 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab029a7b90 con 0x55ab02a4fe80 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.879+0000 7fa48cff9640 10 client.25205 handle_command_reply: tid=0 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.879+0000 7fa4ad68d740 1 client.25205 shutdown 2024-09-19T05:42:21.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.879+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 msgr2=0x55ab02a3dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.879+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a3dd70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009f30 tx=0x7fa4a0011870 comp rx=0 tx=0).stop 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 shutdown_connections 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02a522d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a44650 0x55ab02a420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a3dd70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa48cff9640 0 client.25205 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/1820610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a34f20 0x55ab02a3d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 >> 172.21.15.38:0/1820610520 conn(0x55ab02a315f0 msgr2=0x55ab02a37350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 shutdown_connections 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 1 -- 172.21.15.38:0/1820610520 wait complete. 2024-09-19T05:42:21.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.880+0000 7fa4ad68d740 20 client.25205 trim_cache size 0 max 16384 2024-09-19T05:42:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.909+0000 7fa4ad68d740 1 Processor -- start 2024-09-19T05:42:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.909+0000 7fa4ad68d740 1 -- start start 2024-09-19T05:42:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.909+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.909+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.909+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49394/0 (socket says 172.21.15.38:49394) 2024-09-19T05:42:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 -- 172.21.15.38:0/96757760 learned_addr learned my addr 172.21.15.38:0/96757760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02a57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41acb00 con 0x55ab02a34f20 2024-09-19T05:42:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ae640 con 0x55ab02a35840 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ad1d0 con 0x55ab02a44650 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a9050640 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 -- 172.21.15.38:0/96757760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 msgr2=0x55ab02a57ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02a57ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 -- 172.21.15.38:0/96757760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 msgr2=0x55ab02a30640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a30640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 -- 172.21.15.38:0/96757760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab02a2fbb0 con 0x55ab02a34f20 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a9050640 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.910+0000 7fa4a884f640 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67dbc2df7a7addc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.911+0000 7fa48effd640 1 -- 172.21.15.38:0/96757760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494035040 con 0x55ab02a34f20 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.911+0000 7fa48effd640 1 -- 172.21.15.38:0/96757760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494042040 con 0x55ab02a34f20 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.911+0000 7fa48effd640 1 -- 172.21.15.38:0/96757760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494004ba0 con 0x55ab02a34f20 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.911+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 msgr2=0x55ab02a420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.911+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494031ac0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 shutdown_connections 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02a57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/96757760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a420b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 >> 172.21.15.38:0/96757760 conn(0x55ab02a315f0 msgr2=0x55ab02a2ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 shutdown_connections 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 -- 172.21.15.38:0/96757760 wait complete. 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 -- start start 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.912+0000 7fa4ad68d740 1 Processor -- start 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02ab9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41adf70 con 0x55ab02a34f20 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ad8a0 con 0x55ab02a35840 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41aaae0 con 0x55ab02a44650 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49408/0 (socket says 172.21.15.38:49408) 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 -- 172.21.15.38:0/3928936246 learned_addr learned my addr 172.21.15.38:0/3928936246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4ad68d740 10 client.0 _ll_register_callbacks cb 0x7fa4a0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 -- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 msgr2=0x55ab02ab9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02ab9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 -- 172.21.15.38:0/3928936246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 msgr2=0x55ab02a495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.913+0000 7fa4a884f640 1 -- 172.21.15.38:0/3928936246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab02aba210 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa4a884f640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4940096c0 tx=0x7fa4940319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22bbc3dc1fa95d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494035070 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fa494031bd0 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494042040 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa494031dd0 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa4ad68d740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.914+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab02abb4f0 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494042040 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa49405b040 con 0x55ab02a34f20 2024-09-19T05:42:21.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 10 client.15480 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 20 client.15480 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02abd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 4 client.15480 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bf2a5356", "release": true, "await": true}] 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.915+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bf2a5356", "release": true, "await": true}) -- 0x55ab029a20a0 con 0x55ab02a4fe80 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.916+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02abd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.916+0000 7fa48cff9640 10 client.15480 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:21.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.916+0000 7fa4a9050640 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02abd780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc8796f68199518 server_cookie=c8ef19986f39ce2e in_seq=0 out_seq=0 2024-09-19T05:42:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa48cff9640 1 -- 172.21.15.38:0/3928936246 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ab029a20a0 con 0x55ab02a4fe80 2024-09-19T05:42:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa48cff9640 10 client.15480 handle_command_reply: tid=0 2024-09-19T05:42:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 client.15480 shutdown 2024-09-19T05:42:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 msgr2=0x55ab02a49090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4940096c0 tx=0x7fa4940319c0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 shutdown_connections 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ab02a4fe80 0x55ab02abd780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab02a44650 0x55ab02ab9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab02a35840 0x55ab02a495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 --2- 172.21.15.38:0/3928936246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ab02a34f20 0x55ab02a49090 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 >> 172.21.15.38:0/3928936246 conn(0x55ab02a315f0 msgr2=0x55ab02a30750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa48cff9640 0 client.15480 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 shutdown_connections 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.918+0000 7fa4ad68d740 1 -- 172.21.15.38:0/3928936246 wait complete. 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ad68d740 20 client.15480 trim_cache size 0 max 16384 2024-09-19T05:42:21.922 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 4, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "bf2a5356": { 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 4, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:42:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 131.2, 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:21.924 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:42:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 msgr2=0x7fa4900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 0x7fa4900a5780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0989e0 tx=0x7fa49c0058d0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 msgr2=0x7fa4a419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00eb10 tx=0x7fa49c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.919+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 shutdown_connections 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa4900a32c0 0x7fa4900a5780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a410c1a0 0x7fa4a419cd00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a41036e0 0x7fa4a4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 --2- 172.21.15.38:0/3184262497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a4102dc0 0x7fa4a4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 >> 172.21.15.38:0/3184262497 conn(0x7fa4a40fc7a0 msgr2=0x7fa4a40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 shutdown_connections 2024-09-19T05:42:21.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:21.920+0000 7fa4ab2b4640 1 -- 172.21.15.38:0/3184262497 wait complete. 2024-09-19T05:42:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:21 smithi179 ceph-mon[35781]: pgmap v386: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:22.714 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bf2a5356', total seconds elapsed: 15.2 2024-09-19T05:42:23.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:23 smithi047 ceph-mon[35327]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:23.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:23 smithi179 ceph-mon[35781]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:23.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:23 smithi038 ceph-mon[31227]: pgmap v387: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:42:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:25 smithi047 ceph-mon[35327]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:25.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:25 smithi038 ceph-mon[31227]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:25 smithi179 ceph-mon[35781]: pgmap v388: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:27.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:27 smithi047 ceph-mon[35327]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:27 smithi038 ceph-mon[31227]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:27 smithi179 ceph-mon[35781]: pgmap v389: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:42:29.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:29 smithi047 ceph-mon[35327]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:42:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:29 smithi038 ceph-mon[31227]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:42:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:29 smithi179 ceph-mon[35781]: pgmap v390: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:42:31.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:31 smithi038 ceph-mon[31227]: pgmap v391: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:31 smithi179 ceph-mon[35781]: pgmap v391: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:31.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:31 smithi047 ceph-mon[35327]: pgmap v391: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:33 smithi038 ceph-mon[31227]: pgmap v392: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-19T05:42:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:33 smithi179 ceph-mon[35781]: pgmap v392: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-19T05:42:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:33 smithi047 ceph-mon[35327]: pgmap v392: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-19T05:42:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:35 smithi038 ceph-mon[31227]: pgmap v393: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:35 smithi179 ceph-mon[35781]: pgmap v393: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:35.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:35 smithi047 ceph-mon[35327]: pgmap v393: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:37 smithi038 ceph-mon[31227]: pgmap v394: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:37 smithi179 ceph-mon[35781]: pgmap v394: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:37 smithi047 ceph-mon[35327]: pgmap v394: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-19T05:42:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:39 smithi038 ceph-mon[31227]: pgmap v395: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:42:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:39 smithi179 ceph-mon[35781]: pgmap v395: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:42:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:39 smithi047 ceph-mon[35327]: pgmap v395: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:42:41.695 INFO:tasks.workunit.client.0.smithi038.stderr:Updating files: 94% (12142/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T05:42:41.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:41 smithi038 ceph-mon[31227]: pgmap v396: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:42:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:41 smithi179 ceph-mon[35781]: pgmap v396: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:42:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:41 smithi047 ceph-mon[35327]: pgmap v396: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:42:42.556 INFO:tasks.workunit.client.0.smithi038.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T05:42:42.556 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.556 INFO:tasks.workunit.client.0.smithi038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: git switch -c 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:Or undo this operation with: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: git switch - 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.557 INFO:tasks.workunit.client.0.smithi038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T05:42:42.558 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-19T05:42:42.558 INFO:tasks.workunit.client.0.smithi038.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T05:42:42.562 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T05:42:42.590 INFO:tasks.workunit.client.0.smithi038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T05:42:42.592 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:42:42.592 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T05:42:43.330 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T05:42:43.402 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T05:42:43.445 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:42:43.447 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:42:43.447 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T05:42:43.487 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:42:43.490 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:42:43.490 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T05:42:43.515 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:42:43.515 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T05:42:43.515 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:42:43.549 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-19T05:42:43.550 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-19T05:42:43.550 DEBUG:teuthology.orchestra.run.smithi038:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-19T05:42:43.613 INFO:tasks.workunit.client.0.smithi038.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-19T05:42:43.613 INFO:tasks.workunit.client.0.smithi038.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-19T05:42:43.614 INFO:tasks.workunit.client.0.smithi038.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-19T05:42:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:43.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:43 smithi038 ceph-mon[31227]: pgmap v397: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:43 smithi179 ceph-mon[35781]: pgmap v397: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:43.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:43 smithi047 ceph-mon[35327]: pgmap v397: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:42:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 -- 172.21.15.38:0/1880766890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4103740 msgr2=0x7faab4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 --2- 172.21.15.38:0/1880766890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4103740 0x7faab4107b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009930 tx=0x7faaa802a420 comp rx=0 tx=0).stop 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 -- 172.21.15.38:0/1880766890 shutdown_connections 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 --2- 172.21.15.38:0/1880766890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4108290 0x7faab410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 --2- 172.21.15.38:0/1880766890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4103740 0x7faab4107b80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 --2- 172.21.15.38:0/1880766890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4102d90 0x7faab4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 -- 172.21.15.38:0/1880766890 >> 172.21.15.38:0/1880766890 conn(0x7faab40fc7d0 msgr2=0x7faab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 -- 172.21.15.38:0/1880766890 shutdown_connections 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.371+0000 7faabc454640 1 -- 172.21.15.38:0/1880766890 wait complete. 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 Processor -- start 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 -- start start 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 0x7faab4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 0x7faab4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faab4112090 con 0x7faab4108290 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111f10 con 0x7faab4103740 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faabc454640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faab4112210 con 0x7faab4102d90 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faaba9f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faaba9f1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56684/0 (socket says 172.21.15.38:56684) 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faaba9f1640 1 -- 172.21.15.38:0/2965653396 learned_addr learned my addr 172.21.15.38:0/2965653396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faab99ef640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 0x7faab4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.372+0000 7faaba1f0640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 0x7faab4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 -- 172.21.15.38:0/2965653396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 msgr2=0x7faab4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 0x7faab4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 -- 172.21.15.38:0/2965653396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 msgr2=0x7faab4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 0x7faab4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 -- 172.21.15.38:0/2965653396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab419d540 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faab99ef640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 0x7faab4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba1f0640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 0x7faab4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faaba9f1640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c970 tx=0x7faaac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac007bb0 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac007d50 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab419d830 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac018800 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.373+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab41a5110 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.374+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab4103170 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.375+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faaac00fa50 con 0x7faab4108290 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.376+0000 7faab37fe640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 0x7faa9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.376+0000 7faaba1f0640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 0x7faa9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.376+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7faaac0cfd20 con 0x7faab4108290 2024-09-19T05:42:44.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.376+0000 7faaba1f0640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 0x7faa9c0a5780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faaa4005c40 tx=0x7faaa40082f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.380+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faaac014030 con 0x7faab4108290 2024-09-19T05:42:44.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.592+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faab410a2a0 con 0x7faab4108290 2024-09-19T05:42:44.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.594+0000 7faab37fe640 1 -- 172.21.15.38:0/2965653396 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7faaac09c680 con 0x7faab4108290 2024-09-19T05:42:44.595 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:42:44.595 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:42:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 msgr2=0x7faa9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 0x7faa9c0a5780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faaa4005c40 tx=0x7faaa40082f0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 msgr2=0x7faab419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c970 tx=0x7faaac00ce30 comp rx=0 tx=0).stop 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 shutdown_connections 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faa9c0a32c0 0x7faa9c0a5780 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab4108290 0x7faab419ce60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faab4103740 0x7faab4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 --2- 172.21.15.38:0/2965653396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faab4102d90 0x7faab4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 >> 172.21.15.38:0/2965653396 conn(0x7faab40fc7d0 msgr2=0x7faab4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 shutdown_connections 2024-09-19T05:42:44.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:44.596+0000 7faabc454640 1 -- 172.21.15.38:0/2965653396 wait complete. 2024-09-19T05:42:45.342 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T05:42:45.342 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T05:42:45.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:45 smithi038 ceph-mon[31227]: pgmap v398: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:45.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:45 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2965653396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:45.644 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:42:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:45 smithi179 ceph-mon[35781]: pgmap v398: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:45 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2965653396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:45.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:45 smithi047 ceph-mon[35327]: pgmap v398: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:42:45.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:45 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2965653396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:42:46.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 -- 172.21.15.38:0/2852026520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068102d40 msgr2=0x7f6068103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 --2- 172.21.15.38:0/2852026520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068102d40 0x7f6068103120 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f605802a480 comp rx=0 tx=0).stop 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 -- 172.21.15.38:0/2852026520 shutdown_connections 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 --2- 172.21.15.38:0/2852026520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f606810c170 0x7f606810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 --2- 172.21.15.38:0/2852026520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068103660 0x7f606810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 --2- 172.21.15.38:0/2852026520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068102d40 0x7f6068103120 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 -- 172.21.15.38:0/2852026520 >> 172.21.15.38:0/2852026520 conn(0x7f60680fc7a0 msgr2=0x7f60680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 -- 172.21.15.38:0/2852026520 shutdown_connections 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.146+0000 7f606d6b4640 1 -- 172.21.15.38:0/2852026520 wait complete. 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 Processor -- start 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 -- start start 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 0x7f6068198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606810c170 0x7f606819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6068111f70 con 0x7f6068103660 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6068111df0 con 0x7f6068102d40 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f606d6b4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60681120f0 con 0x7f606810c170 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.147+0000 7f60667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56694/0 (socket says 172.21.15.38:56694) 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 -- 172.21.15.38:0/567240894 learned_addr learned my addr 172.21.15.38:0/567240894 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60677fe640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606810c170 0x7f606819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f6066ffd640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 0x7f6068198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 -- 172.21.15.38:0/567240894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606810c170 msgr2=0x7f606819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606810c170 0x7f606819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 -- 172.21.15.38:0/567240894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 msgr2=0x7f6068198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 0x7f6068198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 -- 172.21.15.38:0/567240894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606819d420 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f6066ffd640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 0x7f6068198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f60667fc640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f605c00d750 tx=0x7f605c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f604ffff640 1 -- 172.21.15.38:0/567240894 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c017070 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f604ffff640 1 -- 172.21.15.38:0/567240894 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f605c005020 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606819d710 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f604ffff640 1 -- 172.21.15.38:0/567240894 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c004d00 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.148+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60681a4ff0 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.151+0000 7f604ffff640 1 -- 172.21.15.38:0/567240894 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f605c010460 con 0x7f6068103660 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.151+0000 7f606fa8d740 1 Processor -- start 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.151+0000 7f606fa8d740 1 -- start start 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564d86703160 con 0x564d8679e750 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564d866aa720 con 0x564d867ade40 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f606fa8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d866ff120 con 0x564d8679f070 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56704/0 (socket says 172.21.15.38:56704) 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 -- 172.21.15.38:0/2464342251 learned_addr learned my addr 172.21.15.38:0/2464342251 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60667fc640 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f6066ffd640 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f604ffff640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 0x7f60500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 -- 172.21.15.38:0/2464342251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 msgr2=0x564d867ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 -- 172.21.15.38:0/2464342251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 msgr2=0x564d867b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 -- 172.21.15.38:0/2464342251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d86474ba0 con 0x564d8679e750 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f604ffff640 1 -- 172.21.15.38:0/567240894 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f605c0cf7c0 con 0x7f6068103660 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 0x7f60500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.152+0000 7f60677fe640 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6060007ec0 tx=0x7f606000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72693469e83d223e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f604d7fa640 1 -- 172.21.15.38:0/2464342251 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060010ac0 con 0x564d8679e750 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f604d7fa640 1 -- 172.21.15.38:0/2464342251 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6060010ce0 con 0x564d8679e750 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f604d7fa640 1 -- 172.21.15.38:0/2464342251 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060019850 con 0x564d8679e750 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 -- 172.21.15.38:0/2464342251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 msgr2=0x564d8679eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f60677fe640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 0x7f60500a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f606001e000 tx=0x7f6060019ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6060007ec0 tx=0x7f606000f790 comp rx=0 tx=0).stop 2024-09-19T05:42:46.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 -- 172.21.15.38:0/2464342251 shutdown_connections 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2464342251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679e750 0x564d8679eb30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.153+0000 7f606fa8d740 1 -- 172.21.15.38:0/2464342251 >> 172.21.15.38:0/2464342251 conn(0x564d8679ade0 msgr2=0x564d8679b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- 172.21.15.38:0/2464342251 shutdown_connections 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- 172.21.15.38:0/2464342251 wait complete. 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- start start 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 Processor -- start 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564d867b0a70 con 0x564d867ade40 2024-09-19T05:42:46.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564d8668b900 con 0x564d8679e750 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d867b0bf0 con 0x564d8679f070 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f606fa8d740 10 client.0 _ll_register_callbacks cb 0x564d86689780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56714/0 (socket says 172.21.15.38:56714) 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f60677fe640 1 -- 172.21.15.38:0/2789307420 learned_addr learned my addr 172.21.15.38:0/2789307420 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f6066ffd640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.154+0000 7f60667fc640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 -- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 msgr2=0x564d867ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 -- 172.21.15.38:0/2789307420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 msgr2=0x564d867aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 -- 172.21.15.38:0/2789307420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d866cbc50 con 0x564d867ade40 2024-09-19T05:42:46.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f6066ffd640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f60677fe640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6060005cc0 tx=0x7f6060015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e68e001b4f289e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f604aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060034060 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f606002c070 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6060010ce0 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f606003b050 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f606fa8d740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d867bc770 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.155+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6060010ce0 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f606000a8d0 con 0x564d867ade40 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:46.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 10 client.15498 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 20 client.15498 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d867bdbd0 0x564d867bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 4 client.15498 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d86710890 con 0x564d867bdbd0 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f6066ffd640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d867bdbd0 0x564d867bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f6066ffd640 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d867bdbd0 0x564d867bffc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a023c66118ddac7e server_cookie=ef4cb0ddeab081c5 in_seq=0 out_seq=0 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.156+0000 7f604aff5640 10 client.15498 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f604aff5640 1 -- 172.21.15.38:0/2789307420 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d86710890 con 0x564d867bdbd0 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f604aff5640 10 client.15498 handle_command_reply: tid=0 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 client.15498 shutdown 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 msgr2=0x564d867ab7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6060005cc0 tx=0x7f6060015000 comp rx=0 tx=0).stop 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 shutdown_connections 2024-09-19T05:42:46.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d867bdbd0 0x564d867bffc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867ab7c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f604aff5640 0 client.15498 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 --2- 172.21.15.38:0/2789307420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 >> 172.21.15.38:0/2789307420 conn(0x564d8679ade0 msgr2=0x564d867a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 shutdown_connections 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.157+0000 7f606fa8d740 1 -- 172.21.15.38:0/2789307420 wait complete. 2024-09-19T05:42:46.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.158+0000 7f606fa8d740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T05:42:46.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.174+0000 7f606fa8d740 1 Processor -- start 2024-09-19T05:42:46.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.174+0000 7f606fa8d740 1 -- start start 2024-09-19T05:42:46.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f60681acb10 con 0x564d867ade40 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f60681ae650 con 0x564d8679e750 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f606fa8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60681ad1e0 con 0x564d8679f070 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56718/0 (socket says 172.21.15.38:56718) 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 -- 172.21.15.38:0/3126772775 learned_addr learned my addr 172.21.15.38:0/3126772775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f6066ffd640 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60667fc640 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 -- 172.21.15.38:0/3126772775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 msgr2=0x564d867c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 -- 172.21.15.38:0/3126772775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 msgr2=0x564d867a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 -- 172.21.15.38:0/3126772775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8679eea0 con 0x564d867ade40 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.175+0000 7f60677fe640 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f60600077c0 tx=0x7f60600021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afb80fb1fb468f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f604cff9640 1 -- 172.21.15.38:0/3126772775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060034040 con 0x564d867ade40 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f604cff9640 1 -- 172.21.15.38:0/3126772775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60600078a0 con 0x564d867ade40 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f604cff9640 1 -- 172.21.15.38:0/3126772775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060010e10 con 0x564d867ade40 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 -- 172.21.15.38:0/3126772775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 msgr2=0x564d867c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f60600077c0 tx=0x7f60600021f0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 -- 172.21.15.38:0/3126772775 shutdown_connections 2024-09-19T05:42:46.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d867ade40 0x564d867c5b30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679f070 0x564d867c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3126772775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d8679e750 0x564d867a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 -- 172.21.15.38:0/3126772775 >> 172.21.15.38:0/3126772775 conn(0x564d8679ade0 msgr2=0x564d867a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 -- 172.21.15.38:0/3126772775 shutdown_connections 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.176+0000 7f606fa8d740 1 -- 172.21.15.38:0/3126772775 wait complete. 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 -- start start 2024-09-19T05:42:46.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 Processor -- start 2024-09-19T05:42:46.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679e750 0x564d867b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f60681adf80 con 0x564d8679f070 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f60681ad8b0 con 0x564d867ade40 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60681aaaf0 con 0x564d8679e750 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f606fa8d740 10 client.0 _ll_register_callbacks cb 0x7f606001d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f60667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f6066ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679e750 0x564d867b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f60667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56732/0 (socket says 172.21.15.38:56732) 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f60667fc640 1 -- 172.21.15.38:0/3903517901 learned_addr learned my addr 172.21.15.38:0/3903517901 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.177+0000 7f60677fe640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 -- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679e750 msgr2=0x564d867b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679e750 0x564d867b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 -- 172.21.15.38:0/3903517901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 msgr2=0x564d867b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 -- 172.21.15.38:0/3903517901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d868275c0 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f60667fc640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f605c09b950 tx=0x7f605c09b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91f5a072e24f9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c020040 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f605c0051d0 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60600078a0 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f605c09fd70 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f606fa8d740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d86828860 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60600078a0 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.178+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f605c0b4bc0 con 0x564d8679f070 2024-09-19T05:42:46.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 10 client.15504 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 20 client.15504 populate_metadata read hostname 'smithi038' 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d86829080 0x564d8682b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 4 client.15504 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x564d86796410 con 0x564d86829080 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f6066ffd640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d86829080 0x564d8682b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f604aff5640 10 client.15504 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:46.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.179+0000 7f6066ffd640 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d86829080 0x564d8682b4a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b221baf4406dec16 server_cookie=964c32486fc4082b in_seq=0 out_seq=0 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f604aff5640 1 -- 172.21.15.38:0/3903517901 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564d86796410 con 0x564d86829080 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f604aff5640 10 client.15504 handle_command_reply: tid=0 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 client.15504 shutdown 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 msgr2=0x564d867b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f605c09b950 tx=0x7f605c09b890 comp rx=0 tx=0).stop 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 shutdown_connections 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x564d86829080 0x564d8682b4a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d867ade40 0x564d867b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d8679f070 0x564d867b3140 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f604aff5640 0 client.15504 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 --2- 172.21.15.38:0/3903517901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8679e750 0x564d867b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 >> 172.21.15.38:0/3903517901 conn(0x564d8679ade0 msgr2=0x564d867acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 shutdown_connections 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.190+0000 7f606fa8d740 1 -- 172.21.15.38:0/3903517901 wait complete. 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606fa8d740 20 client.15504 trim_cache size 0 max 16384 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:42:46.194 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 6, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "fa45ea82": { 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 6, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 140.3, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:42:46.195 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:42:46.196 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 msgr2=0x7f60500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 0x7f60500a5700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f606001e000 tx=0x7f6060019ba0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 msgr2=0x7f60681989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f605c00d750 tx=0x7f605c00dc10 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 shutdown_connections 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f60500a3240 0x7f60500a5700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606810c170 0x7f606819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6068103660 0x7f60681989b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 --2- 172.21.15.38:0/567240894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6068102d40 0x7f6068198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:46.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.191+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 >> 172.21.15.38:0/567240894 conn(0x7f60680fc7a0 msgr2=0x7f60680fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:46.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.192+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 shutdown_connections 2024-09-19T05:42:46.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:42:46.192+0000 7f606d6b4640 1 -- 172.21.15.38:0/567240894 wait complete. 2024-09-19T05:42:46.875 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fa45ea82', quiesce took 3.4 seconds. Will release after: 16.950000000000003 2024-09-19T05:42:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:47 smithi038 ceph-mon[31227]: pgmap v399: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:47 smithi179 ceph-mon[35781]: pgmap v399: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:47 smithi047 ceph-mon[35327]: pgmap v399: 129 pgs: 129 active+clean; 664 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:42:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:49 smithi038 ceph-mon[31227]: pgmap v400: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-19T05:42:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:49 smithi179 ceph-mon[35781]: pgmap v400: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-19T05:42:49.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:49 smithi047 ceph-mon[35327]: pgmap v400: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-19T05:42:51.778 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:51 smithi047 ceph-mon[35327]: pgmap v401: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-19T05:42:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:51 smithi038 ceph-mon[31227]: pgmap v401: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-19T05:42:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:51 smithi179 ceph-mon[35781]: pgmap v401: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-19T05:42:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:53 smithi038 ceph-mon[31227]: pgmap v402: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T05:42:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:53 smithi179 ceph-mon[35781]: pgmap v402: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T05:42:53.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:53 smithi047 ceph-mon[35327]: pgmap v402: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-19T05:42:55.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:55 smithi038 ceph-mon[31227]: pgmap v403: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-19T05:42:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:55 smithi179 ceph-mon[35781]: pgmap v403: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-19T05:42:55.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:55 smithi047 ceph-mon[35327]: pgmap v403: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-19T05:42:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:57 smithi038 ceph-mon[31227]: pgmap v404: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-19T05:42:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:57 smithi179 ceph-mon[35781]: pgmap v404: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-19T05:42:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:57 smithi047 ceph-mon[35327]: pgmap v404: 129 pgs: 129 active+clean; 697 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-09-19T05:42:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:42:59 smithi038 ceph-mon[31227]: pgmap v405: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-19T05:42:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:42:59 smithi179 ceph-mon[35781]: pgmap v405: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-19T05:42:59.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:42:59 smithi047 ceph-mon[35327]: pgmap v405: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-19T05:43:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:01 smithi038 ceph-mon[31227]: pgmap v406: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:01 smithi179 ceph-mon[35781]: pgmap v406: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:01.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:01 smithi047 ceph-mon[35327]: pgmap v406: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:03.826 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:43:03.855 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:03 smithi038 ceph-mon[31227]: pgmap v407: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:43:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:03 smithi179 ceph-mon[35781]: pgmap v407: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:43:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:03 smithi047 ceph-mon[35327]: pgmap v407: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:43:04.098 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:04.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 -- 172.21.15.38:0/1351993899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 msgr2=0x7f94ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1351993899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac10ea90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ca0 tx=0x7f94a402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 -- 172.21.15.38:0/1351993899 shutdown_connections 2024-09-19T05:43:04.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1351993899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac10ea90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1351993899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 0x7f94ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1351993899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 0x7f94ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 -- 172.21.15.38:0/1351993899 >> 172.21.15.38:0/1351993899 conn(0x7f94ac0fd8c0 msgr2=0x7f94ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 -- 172.21.15.38:0/1351993899 shutdown_connections 2024-09-19T05:43:04.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.590+0000 7f94b3d49640 1 -- 172.21.15.38:0/1351993899 wait complete. 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 Processor -- start 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 -- start start 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 0x7f94ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 0x7f94ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac111fe0 con 0x7f94ac106f10 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac111e60 con 0x7f94ac101de0 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.591+0000 7f94b3d49640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac112160 con 0x7f94ac102790 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48262/0 (socket says 172.21.15.38:48262) 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 -- 172.21.15.38:0/1893194957 learned_addr learned my addr 172.21.15.38:0/1893194957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b1ae5640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 0x7f94ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b12e4640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 0x7f94ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 -- 172.21.15.38:0/1893194957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 msgr2=0x7f94ac1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 0x7f94ac1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 -- 172.21.15.38:0/1893194957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 msgr2=0x7f94ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 0x7f94ac198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 -- 172.21.15.38:0/1893194957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac19d430 con 0x7f94ac106f10 2024-09-19T05:43:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b22e6640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f94a4003f40 tx=0x7f94a4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a40318c0 con 0x7f94ac106f10 2024-09-19T05:43:04.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a4004520 con 0x7f94ac106f10 2024-09-19T05:43:04.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac19d6c0 con 0x7f94ac106f10 2024-09-19T05:43:04.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a402fe10 con 0x7f94ac106f10 2024-09-19T05:43:04.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.592+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a5000 con 0x7f94ac106f10 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.594+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac1021c0 con 0x7f94ac106f10 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.595+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f94a40046d0 con 0x7f94ac106f10 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.595+0000 7f94a2ffd640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 0x7f94980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.595+0000 7f94b1ae5640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 0x7f94980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.595+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f94a40f1350 con 0x7f94ac106f10 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.596+0000 7f94b1ae5640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 0x7f94980a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94a80058c0 tx=0x7f94a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.600+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f94a40bdcb0 con 0x7f94ac106f10 2024-09-19T05:43:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.811+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94ac10a0f0 con 0x7f94ac106f10 2024-09-19T05:43:04.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.812+0000 7f94a2ffd640 1 -- 172.21.15.38:0/1893194957 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f94a40bd5b0 con 0x7f94ac106f10 2024-09-19T05:43:04.815 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:43:04.815 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:43:04.816 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:43:04.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 msgr2=0x7f94980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 0x7f94980a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94a80058c0 tx=0x7f94a800a2d0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 msgr2=0x7f94ac19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f94a4003f40 tx=0x7f94a4003f70 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 shutdown_connections 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94980a3240 0x7f94980a5700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94ac106f10 0x7f94ac19cd50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94ac102790 0x7f94ac1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 --2- 172.21.15.38:0/1893194957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94ac101de0 0x7f94ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 >> 172.21.15.38:0/1893194957 conn(0x7f94ac0fd8c0 msgr2=0x7f94ac0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:04.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 shutdown_connections 2024-09-19T05:43:04.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:04.814+0000 7f94b3d49640 1 -- 172.21.15.38:0/1893194957 wait complete. 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: pgmap v408: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1893194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.768 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.769 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:05 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.795 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id fa45ea82 --release --await' 2024-09-19T05:43:05.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id fa45ea82 --release --await 2024-09-19T05:43:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: pgmap v408: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1893194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:05 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: pgmap v408: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1893194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:05 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 -- 172.21.15.38:0/3847235815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 msgr2=0x7f04601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 --2- 172.21.15.38:0/3847235815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f04601067e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0448009900 tx=0x7f044802a410 comp rx=0 tx=0).stop 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 -- 172.21.15.38:0/3847235815 shutdown_connections 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 --2- 172.21.15.38:0/3847235815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 0x7f046010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 --2- 172.21.15.38:0/3847235815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f04601067e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 --2- 172.21.15.38:0/3847235815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 0x7f04601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 -- 172.21.15.38:0/3847235815 >> 172.21.15.38:0/3847235815 conn(0x7f04600fd8c0 msgr2=0x7f04600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 -- 172.21.15.38:0/3847235815 shutdown_connections 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.619+0000 7f046660f640 1 -- 172.21.15.38:0/3847235815 wait complete. 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 Processor -- start 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 -- start start 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 0x7f04600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 0x7f0460074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0460111f70 con 0x7f0460102790 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0460111df0 con 0x7f0460101de0 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f046660f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04601120f0 con 0x7f0460106f10 2024-09-19T05:43:06.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48276/0 (socket says 172.21.15.38:48276) 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 -- 172.21.15.38:0/3287636314 learned_addr learned my addr 172.21.15.38:0/3287636314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f045ffff640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 0x7f04600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0464bac640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 0x7f0460074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 -- 172.21.15.38:0/3287636314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 msgr2=0x7f0460074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 0x7f0460074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 -- 172.21.15.38:0/3287636314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 msgr2=0x7f04600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 0x7f04600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.620+0000 7f0457fff640 1 -- 172.21.15.38:0/3287636314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04601a68c0 con 0x7f0460102790 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f0457fff640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0448006010 tx=0x7f04480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f045dffb640 1 -- 172.21.15.38:0/3287636314 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044803f030 con 0x7f0460102790 2024-09-19T05:43:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a6b50 con 0x7f0460102790 2024-09-19T05:43:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f045dffb640 1 -- 172.21.15.38:0/3287636314 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448003ce0 con 0x7f0460102790 2024-09-19T05:43:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f045dffb640 1 -- 172.21.15.38:0/3287636314 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448035070 con 0x7f0460102790 2024-09-19T05:43:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.621+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a7080 con 0x7f0460102790 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.623+0000 7f045dffb640 1 -- 172.21.15.38:0/3287636314 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0448042030 con 0x7f0460102790 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.623+0000 7f04689e8740 1 Processor -- start 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 -- start start 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 0x5622ef7e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 0x5622ef7e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5622ef73e620 con 0x5622ef7d9060 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5622ef6e5e50 con 0x5622ef7e4240 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f04689e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5622ef73a5e0 con 0x5622ef7d9980 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48292/0 (socket says 172.21.15.38:48292) 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 -- 172.21.15.38:0/2045280721 learned_addr learned my addr 172.21.15.38:0/2045280721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f045ffff640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 0x5622ef7e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0457fff640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 0x5622ef7e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f045dffb640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 0x7f044c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 -- 172.21.15.38:0/2045280721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 msgr2=0x5622ef7e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 0x5622ef7e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 -- 172.21.15.38:0/2045280721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 msgr2=0x5622ef7e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 0x5622ef7e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0464bac640 1 -- 172.21.15.38:0/2045280721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ef6d6d40 con 0x5622ef7d9060 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f045ffff640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 0x5622ef7e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.624+0000 7f0457fff640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 0x5622ef7e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f0464bac640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 0x7f044c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f045dffb640 1 -- 172.21.15.38:0/3287636314 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f04480f0190 con 0x7f0460102790 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f0464bac640 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0458009fc0 tx=0x7f0458010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d5aacba42b351a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f04567fc640 1 -- 172.21.15.38:0/2045280721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458023070 con 0x5622ef7d9060 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f04567fc640 1 -- 172.21.15.38:0/2045280721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045801cd90 con 0x5622ef7d9060 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.625+0000 7f04567fc640 1 -- 172.21.15.38:0/2045280721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800c360 con 0x5622ef7d9060 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- 172.21.15.38:0/2045280721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 msgr2=0x5622ef7d9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f0464bac640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 0x7f044c0a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0458020000 tx=0x7f045801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0458009fc0 tx=0x7f0458010800 comp rx=0 tx=0).stop 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- 172.21.15.38:0/2045280721 shutdown_connections 2024-09-19T05:43:06.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7e4240 0x5622ef7e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9980 0x5622ef7e3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 --2- 172.21.15.38:0/2045280721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7d9440 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- 172.21.15.38:0/2045280721 >> 172.21.15.38:0/2045280721 conn(0x5622ef7d55f0 msgr2=0x5622ef7d59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- 172.21.15.38:0/2045280721 shutdown_connections 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- 172.21.15.38:0/2045280721 wait complete. 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 -- start start 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.626+0000 7f04689e8740 1 Processor -- start 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5622ef7e6e10 con 0x5622ef7d9060 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5622ef6c6f10 con 0x5622ef7d9980 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5622ef7e6f90 con 0x5622ef7e4240 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f04689e8740 10 client.0 _ll_register_callbacks cb 0x5622ef6eadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48296/0 (socket says 172.21.15.38:48296) 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 -- 172.21.15.38:0/3991820598 learned_addr learned my addr 172.21.15.38:0/3991820598 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f0457fff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f0464bac640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 -- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 msgr2=0x5622ef7e1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7e1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 -- 172.21.15.38:0/3991820598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 msgr2=0x5622ef7e16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 -- 172.21.15.38:0/3991820598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ef44d580 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f045ffff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0450010820 tx=0x7f0450010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e545b15af006030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.627+0000 7f043f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045000fb90 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f045000fd30 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045801cd90 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0450024050 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622ef517960 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045801cd90 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f045000e5e0 con 0x5622ef7d9060 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 10 client.15522 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 20 client.15522 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef7f3e90 0x5622ef7f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 4 client.15522 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.628+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ef74bb90 con 0x5622ef7f3e90 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.629+0000 7f0457fff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef7f3e90 0x5622ef7f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.629+0000 7f0457fff640 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef7f3e90 0x5622ef7f62f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b3d48acdc1a7bd server_cookie=be46840515f3ca75 in_seq=0 out_seq=0 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.629+0000 7f043f7fe640 10 client.15522 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f043f7fe640 1 -- 172.21.15.38:0/3991820598 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622ef74bb90 con 0x5622ef7f3e90 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f043f7fe640 10 client.15522 handle_command_reply: tid=0 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 client.15522 shutdown 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 msgr2=0x5622ef7e11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0450010820 tx=0x7f0450010ce0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 shutdown_connections 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef7f3e90 0x5622ef7f62f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f043f7fe640 0 client.15522 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 --2- 172.21.15.38:0/3991820598 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9060 0x5622ef7e11a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 >> 172.21.15.38:0/3991820598 conn(0x5622ef7d55f0 msgr2=0x5622ef7db2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 shutdown_connections 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 1 -- 172.21.15.38:0/3991820598 wait complete. 2024-09-19T05:43:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.630+0000 7f04689e8740 20 client.15522 trim_cache size 0 max 16384 2024-09-19T05:43:06.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.647+0000 7f04689e8740 1 Processor -- start 2024-09-19T05:43:06.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.647+0000 7f04689e8740 1 -- start start 2024-09-19T05:43:06.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 0x5622ef7d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04601b0f40 con 0x5622ef7d9980 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04601b2a80 con 0x5622ef7d9060 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f04689e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04601b1610 con 0x5622ef7e4240 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48312/0 (socket says 172.21.15.38:48312) 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 -- 172.21.15.38:0/2384532259 learned_addr learned my addr 172.21.15.38:0/2384532259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f045ffff640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 0x5622ef7d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0464bac640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 -- 172.21.15.38:0/2384532259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 msgr2=0x5622ef7fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 -- 172.21.15.38:0/2384532259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 msgr2=0x5622ef7d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 0x5622ef7d59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 -- 172.21.15.38:0/2384532259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ef7e0150 con 0x5622ef7d9980 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f045ffff640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 0x5622ef7d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0464bac640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0457fff640 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0448005ab0 tx=0x7f044802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977f06cf4b368d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0455ffb640 1 -- 172.21.15.38:0/2384532259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448049040 con 0x5622ef7d9980 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0455ffb640 1 -- 172.21.15.38:0/2384532259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044803b580 con 0x5622ef7d9980 2024-09-19T05:43:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.648+0000 7f0455ffb640 1 -- 172.21.15.38:0/2384532259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480ba7b0 con 0x5622ef7d9980 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- 172.21.15.38:0/2384532259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 msgr2=0x5622ef7d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0448005ab0 tx=0x7f044802a410 comp rx=0 tx=0).stop 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- 172.21.15.38:0/2384532259 shutdown_connections 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7e4240 0x5622ef7fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7d9980 0x5622ef7d4640 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 --2- 172.21.15.38:0/2384532259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9060 0x5622ef7d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- 172.21.15.38:0/2384532259 >> 172.21.15.38:0/2384532259 conn(0x5622ef7d55f0 msgr2=0x5622ef7d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- 172.21.15.38:0/2384532259 shutdown_connections 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- 172.21.15.38:0/2384532259 wait complete. 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.649+0000 7f04689e8740 1 -- start start 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 Processor -- start 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9060 0x5622ef7ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04601b23b0 con 0x5622ef7e4240 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04601b1ce0 con 0x5622ef7d9980 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04601aef20 con 0x5622ef7d9060 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f04689e8740 10 client.0 _ll_register_callbacks cb 0x7f0450010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f0464bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f0464bac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48326/0 (socket says 172.21.15.38:48326) 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f0464bac640 1 -- 172.21.15.38:0/2866750602 learned_addr learned my addr 172.21.15.38:0/2866750602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f0457fff640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.650+0000 7f045ffff640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9060 0x5622ef7ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 -- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9060 msgr2=0x5622ef7ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9060 0x5622ef7ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 -- 172.21.15.38:0/2866750602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 msgr2=0x5622ef7e91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 -- 172.21.15.38:0/2866750602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ef7e9d60 con 0x5622ef7e4240 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f0464bac640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0458000f80 tx=0x7f0458013270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba29377b67510858 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f043f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045802d040 con 0x5622ef7e4240 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f045801ca30 con 0x5622ef7e4240 2024-09-19T05:43:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5622ef7460a0 con 0x5622ef7e4240 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f045803f050 con 0x5622ef7e4240 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f04689e8740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622ef85faa0 con 0x5622ef7e4240 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.651+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5622ef7460a0 con 0x5622ef7e4240 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f045800f040 con 0x5622ef7e4240 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 10 client.15528 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 20 client.15528 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef860040 0x5622ef862460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 4 client.15528 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fa45ea82", "release": true, "await": true}] 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fa45ea82", "release": true, "await": true}) -- 0x5622ef6c25a0 con 0x5622ef860040 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f045ffff640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef860040 0x5622ef862460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f043f7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.652+0000 7f045ffff640 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef860040 0x5622ef862460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b63feb3567c8b3 server_cookie=ed0cbad0894a770f in_seq=0 out_seq=0 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f043f7fe640 1 -- 172.21.15.38:0/2866750602 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5622ef6c25a0 con 0x5622ef860040 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f043f7fe640 10 client.15528 handle_command_reply: tid=0 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 client.15528 shutdown 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 msgr2=0x5622ef7e96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0458000f80 tx=0x7f0458013270 comp rx=0 tx=0).stop 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 shutdown_connections 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5622ef860040 0x5622ef862460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f043f7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5622ef7e4240 0x5622ef7e96f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622ef7d9980 0x5622ef7e91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 --2- 172.21.15.38:0/2866750602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622ef7d9060 0x5622ef7ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 >> 172.21.15.38:0/2866750602 conn(0x5622ef7d55f0 msgr2=0x5622ef7d4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 shutdown_connections 2024-09-19T05:43:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 1 -- 172.21.15.38:0/2866750602 wait complete. 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.655+0000 7f04689e8740 20 client.15528 trim_cache size 0 max 16384 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 8, 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "fa45ea82": { 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 8, 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:43:06.663 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 140.3, 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:06.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 msgr2=0x7f044c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 0x7f044c0a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0458020000 tx=0x7f045801d000 comp rx=0 tx=0).stop 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 msgr2=0x7f0460074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0448006010 tx=0x7f04480042f0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 shutdown_connections 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f044c0a32c0 0x7f044c0a5780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0460106f10 0x7f0460074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0460102790 0x7f0460074a30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 --2- 172.21.15.38:0/3287636314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0460101de0 0x7f04600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 >> 172.21.15.38:0/3287636314 conn(0x7f04600fd8c0 msgr2=0x7f04600fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 shutdown_connections 2024-09-19T05:43:06.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:06.656+0000 7f046660f640 1 -- 172.21.15.38:0/3287636314 wait complete. 2024-09-19T05:43:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:43:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:43:06.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:43:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:43:06.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:06.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:43:06.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:43:06.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:43:07.468 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fa45ea82', total seconds elapsed: 24.0 2024-09-19T05:43:07.468 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T05:43:07.469 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:43:07.730 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:07.852 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:07 smithi038 ceph-mon[31227]: pgmap v409: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:07 smithi179 ceph-mon[35781]: pgmap v409: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:07.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:07 smithi047 ceph-mon[35327]: pgmap v409: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 -- 172.21.15.38:0/1722600652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100e30 msgr2=0x7f8a08104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 --2- 172.21.15.38:0/1722600652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100e30 0x7f8a08104e80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f802a410 comp rx=0 tx=0).stop 2024-09-19T05:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 -- 172.21.15.38:0/1722600652 shutdown_connections 2024-09-19T05:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 --2- 172.21.15.38:0/1722600652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a0810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 --2- 172.21.15.38:0/1722600652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100e30 0x7f8a08104e80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 --2- 172.21.15.38:0/1722600652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100480 0x7f8a08100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 -- 172.21.15.38:0/1722600652 >> 172.21.15.38:0/1722600652 conn(0x7f8a08077210 msgr2=0x7f8a080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 -- 172.21.15.38:0/1722600652 shutdown_connections 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.203+0000 7f8a0f566640 1 -- 172.21.15.38:0/1722600652 wait complete. 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 Processor -- start 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 -- start start 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100e30 0x7f8a081998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a081a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08110630 con 0x7f8a08100480 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081104b0 con 0x7f8a08100e30 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0f566640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081107b0 con 0x7f8a081055b0 2024-09-19T05:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.204+0000 7f8a0d302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50386/0 (socket says 172.21.15.38:50386) 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 -- 172.21.15.38:0/716732280 learned_addr learned my addr 172.21.15.38:0/716732280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0cb01640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100e30 0x7f8a081998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0db03640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a081a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 -- 172.21.15.38:0/716732280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 msgr2=0x7f8a081a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a081a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 -- 172.21.15.38:0/716732280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100e30 msgr2=0x7f8a081998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100e30 0x7f8a081998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 -- 172.21.15.38:0/716732280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a4970 con 0x7f8a08100480 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0db03640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a081a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0d302640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c890 tx=0x7f8a0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004220 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a040043c0 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a4b40 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040049e0 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.205+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a5000 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.206+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a08100860 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.207+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8a040079b0 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.208+0000 7f89fe7fc640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 0x7f89f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.208+0000 7f8a0cb01640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 0x7f89f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.208+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f8a040cfd30 con 0x7f8a08100480 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.209+0000 7f8a0cb01640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 0x7f89f40a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000c00 tx=0x7f89f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.212+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a04014030 con 0x7f8a08100480 2024-09-19T05:43:08.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.423+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a081a5260 con 0x7f8a08100480 2024-09-19T05:43:08.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.424+0000 7f89fe7fc640 1 -- 172.21.15.38:0/716732280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f8a0409c690 con 0x7f8a08100480 2024-09-19T05:43:08.427 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:43:08.427 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:43:08.427 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 msgr2=0x7f89f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 0x7f89f40a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000c00 tx=0x7f89f80058d0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 msgr2=0x7f8a081993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c890 tx=0x7f8a0400cd50 comp rx=0 tx=0).stop 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 shutdown_connections 2024-09-19T05:43:08.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89f40a3240 0x7f89f40a5700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a081055b0 0x7f8a081a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a08100e30 0x7f8a081998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 --2- 172.21.15.38:0/716732280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a08100480 0x7f8a081993b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.426+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 >> 172.21.15.38:0/716732280 conn(0x7f8a08077210 msgr2=0x7f8a080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.427+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 shutdown_connections 2024-09-19T05:43:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:08.427+0000 7f8a0f566640 1 -- 172.21.15.38:0/716732280 wait complete. 2024-09-19T05:43:08.865 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:08 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/716732280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:08 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/716732280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:08.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:08 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/716732280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:09.104 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T05:43:09.104 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T05:43:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:09.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:09 smithi038 ceph-mon[31227]: pgmap v410: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 -- 172.21.15.38:0/2103956968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed010c1a0 msgr2=0x7f5ed010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 --2- 172.21.15.38:0/2103956968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed010c1a0 0x7f5ed010e610 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009ca0 tx=0x7f5ec802a950 comp rx=0 tx=0).stop 2024-09-19T05:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 -- 172.21.15.38:0/2103956968 shutdown_connections 2024-09-19T05:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 --2- 172.21.15.38:0/2103956968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed010c1a0 0x7f5ed010e610 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 --2- 172.21.15.38:0/2103956968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 0x7f5ed010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 --2- 172.21.15.38:0/2103956968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed0102dc0 0x7f5ed01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 -- 172.21.15.38:0/2103956968 >> 172.21.15.38:0/2103956968 conn(0x7f5ed00fc7a0 msgr2=0x7f5ed00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 -- 172.21.15.38:0/2103956968 shutdown_connections 2024-09-19T05:43:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.866+0000 7f5ed4948640 1 -- 172.21.15.38:0/2103956968 wait complete. 2024-09-19T05:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 Processor -- start 2024-09-19T05:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 -- start start 2024-09-19T05:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 0x7f5ed0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed010c1a0 0x7f5ed019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0111f30 con 0x7f5ed0102dc0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0111db0 con 0x7f5ed010c1a0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.867+0000 7f5ed4948640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01120b0 con 0x7f5ed01036e0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50414/0 (socket says 172.21.15.38:50414) 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 -- 172.21.15.38:0/287917414 learned_addr learned my addr 172.21.15.38:0/287917414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 0x7f5ed0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5eced9d640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed010c1a0 0x7f5ed019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 -- 172.21.15.38:0/287917414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 msgr2=0x7f5ed0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 0x7f5ed0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 -- 172.21.15.38:0/287917414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed010c1a0 msgr2=0x7f5ed019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed010c1a0 0x7f5ed019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 -- 172.21.15.38:0/287917414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed019d3e0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ece59c640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000d950 tx=0x7f5ec000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ec77fe640 1 -- 172.21.15.38:0/287917414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0017030 con 0x7f5ed0102dc0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed019d6d0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ec77fe640 1 -- 172.21.15.38:0/287917414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0004020 con 0x7f5ed0102dc0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ec77fe640 1 -- 172.21.15.38:0/287917414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00047a0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.868+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed01a4fb0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.871+0000 7f5ec77fe640 1 -- 172.21.15.38:0/287917414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5ec00041d0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.871+0000 7f5ed6d21740 1 Processor -- start 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.871+0000 7f5ed6d21740 1 -- start start 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec77fe640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 0x7f5ebc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee300a30 0x55e7ee302e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee25a160 con 0x55e7ee2f6170 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee201720 con 0x55e7ee2f5850 2024-09-19T05:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee256120 con 0x55e7ee300a30 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5eced9d640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 0x7f5ebc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50422/0 (socket says 172.21.15.38:50422) 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 -- 172.21.15.38:0/3017667763 learned_addr learned my addr 172.21.15.38:0/3017667763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec77fe640 1 -- 172.21.15.38:0/287917414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5ec00cf5e0 con 0x7f5ed0102dc0 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ece59c640 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5eced9d640 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee300a30 0x55e7ee302e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 -- 172.21.15.38:0/3017667763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee300a30 msgr2=0x55e7ee302e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee300a30 0x55e7ee302e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 -- 172.21.15.38:0/3017667763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 msgr2=0x55e7ee2f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 -- 172.21.15.38:0/3017667763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7edfcbba0 con 0x55e7ee2f6170 2024-09-19T05:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5eced9d640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 0x7f5ebc0a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005e30 tx=0x7f5ec802fa20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.872+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8012c80 tx=0x7f5eb800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f55a571a285507 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ec4ff9640 1 -- 172.21.15.38:0/3017667763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8020070 con 0x55e7ee2f6170 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ec4ff9640 1 -- 172.21.15.38:0/3017667763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb801b8f0 con 0x55e7ee2f6170 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ec4ff9640 1 -- 172.21.15.38:0/3017667763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800da00 con 0x55e7ee2f6170 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 -- 172.21.15.38:0/3017667763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 msgr2=0x55e7ee3004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8012c80 tx=0x7f5eb800ba60 comp rx=0 tx=0).stop 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 -- 172.21.15.38:0/3017667763 shutdown_connections 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee300a30 0x55e7ee302e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee3004f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/3017667763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.873+0000 7f5ed6d21740 1 -- 172.21.15.38:0/3017667763 >> 172.21.15.38:0/3017667763 conn(0x55e7ee2f1de0 msgr2=0x55e7ee2f21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- 172.21.15.38:0/3017667763 shutdown_connections 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- 172.21.15.38:0/3017667763 wait complete. 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- start start 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 Processor -- start 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee2fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee303600 con 0x55e7ee2f5850 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee1e2900 con 0x55e7ee300a30 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee303780 con 0x55e7ee2f6170 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ed6d21740 10 client.0 _ll_register_callbacks cb 0x55e7ee1e0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50432/0 (socket says 172.21.15.38:50432) 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5ece59c640 1 -- 172.21.15.38:0/1778051019 learned_addr learned my addr 172.21.15.38:0/1778051019 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.874+0000 7f5eced9d640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee2fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 -- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 msgr2=0x55e7ee2fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee2fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 -- 172.21.15.38:0/1778051019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 msgr2=0x55e7ee2fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 -- 172.21.15.38:0/1778051019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ee222c50 con 0x55e7ee2f5850 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee2fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ece59c640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0097ca0 tx=0x7f5ec009b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904412988933c91b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00a0d80 con 0x55e7ee2f5850 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5ec0099b40 con 0x55e7ee2f5850 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb801b8f0 con 0x55e7ee2f5850 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ec009f120 con 0x55e7ee2f5850 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ed6d21740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7ee30f2e0 con 0x55e7ee2f5850 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.875+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb801b8f0 con 0x55e7ee2f5850 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5ec0097eb0 con 0x55e7ee2f5850 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 10 client.15546 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 20 client.15546 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee310790 0x55e7ee312bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 4 client.15546 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ee267890 con 0x55e7ee310790 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee310790 0x55e7ee312bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee310790 0x55e7ee312bc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3568c51c0fd5c7 server_cookie=7b53c5af8464cdd in_seq=0 out_seq=0 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.876+0000 7f5eb5ffb640 10 client.15546 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/1778051019 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ee267890 con 0x55e7ee310790 2024-09-19T05:43:09.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5eb5ffb640 10 client.15546 handle_command_reply: tid=0 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 client.15546 shutdown 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 msgr2=0x55e7ee2fd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0097ca0 tx=0x7f5ec009b210 comp rx=0 tx=0).stop 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 shutdown_connections 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee310790 0x55e7ee312bc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee2fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1778051019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f5850 0x55e7ee2fd8e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.877+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 >> 172.21.15.38:0/1778051019 conn(0x55e7ee2f1de0 msgr2=0x55e7ee2f7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.878+0000 7f5eb5ffb640 0 client.15546 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.878+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 shutdown_connections 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.878+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1778051019 wait complete. 2024-09-19T05:43:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.878+0000 7f5ed6d21740 20 client.15546 trim_cache size 0 max 16384 2024-09-19T05:43:09.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 Processor -- start 2024-09-19T05:43:09.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 -- start start 2024-09-19T05:43:09.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee2f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01acb30 con 0x55e7ee2f6170 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01ae670 con 0x55e7ee300a30 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.895+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01ad200 con 0x55e7ee2f5850 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ece59c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:42754/0 (socket says 172.21.15.38:42754) 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ece59c640 1 -- 172.21.15.38:0/1196299188 learned_addr learned my addr 172.21.15.38:0/1196299188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee2f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5eced9d640 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 -- 172.21.15.38:0/1196299188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 msgr2=0x55e7ee2f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 -- 172.21.15.38:0/1196299188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 msgr2=0x55e7ee2f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 -- 172.21.15.38:0/1196299188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ee2f5fa0 con 0x55e7ee2f6170 2024-09-19T05:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.896+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee2f0db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801e040 tx=0x7f5eb800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526dfa161ac4e782 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5eb7fff640 1 -- 172.21.15.38:0/1196299188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8020040 con 0x55e7ee2f6170 2024-09-19T05:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5eb7fff640 1 -- 172.21.15.38:0/1196299188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb8021040 con 0x55e7ee2f6170 2024-09-19T05:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5eb7fff640 1 -- 172.21.15.38:0/1196299188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8012070 con 0x55e7ee2f6170 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1196299188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 msgr2=0x55e7ee2f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee2f0db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5eb801e040 tx=0x7f5eb800b5c0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1196299188 shutdown_connections 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee300a30 0x55e7ee2f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee2f6170 0x55e7ee2f0db0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.897+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/1196299188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f5850 0x55e7ee2f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1196299188 >> 172.21.15.38:0/1196299188 conn(0x55e7ee2f1de0 msgr2=0x55e7ee2f06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1196299188 shutdown_connections 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- 172.21.15.38:0/1196299188 wait complete. 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- start start 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 Processor -- start 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee3049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee304f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 0x55e7ee305440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01adfa0 con 0x55e7ee300a30 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01ad8d0 con 0x55e7ee2f5850 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01aab10 con 0x55e7ee2f6170 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.898+0000 7f5ed6d21740 10 client.0 _ll_register_callbacks cb 0x7f5ec0097ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5eced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 0x55e7ee305440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5eced9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 0x55e7ee305440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50450/0 (socket says 172.21.15.38:50450) 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5eced9d640 1 -- 172.21.15.38:0/120939401 learned_addr learned my addr 172.21.15.38:0/120939401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee304f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ece59c640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee3049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 -- 172.21.15.38:0/120939401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 msgr2=0x55e7ee3049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee3049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 -- 172.21.15.38:0/120939401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 msgr2=0x55e7ee305440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 0x55e7ee305440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.899+0000 7f5ec7fff640 1 -- 172.21.15.38:0/120939401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ee382ec0 con 0x55e7ee2f6170 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5ec7fff640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee304f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000c00 tx=0x7f5eb800b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b804b7c19b8f28a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5eb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8037070 con 0x55e7ee2f6170 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f5eb8004020 con 0x55e7ee2f6170 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb8021040 con 0x55e7ee2f6170 2024-09-19T05:43:09.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8043050 con 0x55e7ee2f6170 2024-09-19T05:43:09.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5ed6d21740 10 client.25279 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:09.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.900+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7ee384180 con 0x55e7ee2f6170 2024-09-19T05:43:09.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb8021040 con 0x55e7ee2f6170 2024-09-19T05:43:09.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5eb8028070 con 0x55e7ee2f6170 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 10 client.25279 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 10 client.25279 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 20 client.25279 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee384690 0x55e7ee386ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 4 client.25279 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x55e7ee261be0 con 0x55e7ee384690 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.901+0000 7f5ece59c640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee384690 0x55e7ee386ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.902+0000 7f5eb5ffb640 10 client.25279 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:09.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.902+0000 7f5ece59c640 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee384690 0x55e7ee386ab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37add4548be75a56 server_cookie=d7ccd56eb270ecef in_seq=0 out_seq=0 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.913+0000 7f5eb5ffb640 1 -- 172.21.15.38:0/120939401 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f5ec0014260 con 0x55e7ee384690 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5eb5ffb640 10 client.25279 handle_command_reply: tid=0 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 client.25279 shutdown 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 msgr2=0x55e7ee304f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee304f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000c00 tx=0x7f5eb800b4a0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 shutdown_connections 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e7ee384690 0x55e7ee386ab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7ee300a30 0x55e7ee305440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7ee2f6170 0x55e7ee304f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5eb5ffb640 0 client.25279 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 --2- 172.21.15.38:0/120939401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ee2f5850 0x55e7ee3049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 >> 172.21.15.38:0/120939401 conn(0x55e7ee2f1de0 msgr2=0x55e7ee2f0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 shutdown_connections 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 1 -- 172.21.15.38:0/120939401 wait complete. 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed6d21740 20 client.25279 trim_cache size 0 max 16384 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 10, 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:43:09.918 INFO:teuthology.orchestra.run.smithi038.stdout: "52e7bead": { 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 10, 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 140.3, 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:43:09.919 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:09.920 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 msgr2=0x7f5ebc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 0x7f5ebc0a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005e30 tx=0x7f5ec802fa20 comp rx=0 tx=0).stop 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.914+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 msgr2=0x7f5ed0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000d950 tx=0x7f5ec000de10 comp rx=0 tx=0).stop 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 shutdown_connections 2024-09-19T05:43:09.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f5ebc0a3240 0x7f5ebc0a5700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ed010c1a0 0x7f5ed019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed01036e0 0x7f5ed0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 --2- 172.21.15.38:0/287917414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ed0102dc0 0x7f5ed0198430 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 >> 172.21.15.38:0/287917414 conn(0x7f5ed00fc7a0 msgr2=0x7f5ed00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 shutdown_connections 2024-09-19T05:43:09.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:09.915+0000 7f5ed4948640 1 -- 172.21.15.38:0/287917414 wait complete. 2024-09-19T05:43:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:09 smithi179 ceph-mon[35781]: pgmap v410: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:09.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:09 smithi047 ceph-mon[35327]: pgmap v410: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:10.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:10 smithi038 ceph-mon[31227]: pgmap v411: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:43:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:10 smithi179 ceph-mon[35781]: pgmap v411: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:43:10.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:10 smithi047 ceph-mon[35327]: pgmap v411: 129 pgs: 129 active+clean; 702 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:43:11.142 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '52e7bead', quiesce took 3.7 seconds. Will release after: 16.650000000000002 2024-09-19T05:43:12.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:12 smithi047 ceph-mon[35327]: pgmap v412: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:43:13.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:12 smithi038 ceph-mon[31227]: pgmap v412: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:43:13.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:12 smithi179 ceph-mon[35781]: pgmap v412: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:43:14.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:14 smithi047 ceph-mon[35327]: pgmap v413: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:15.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:14 smithi038 ceph-mon[31227]: pgmap v413: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:14 smithi179 ceph-mon[35781]: pgmap v413: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:16 smithi047 ceph-mon[35327]: pgmap v414: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:16 smithi038 ceph-mon[31227]: pgmap v414: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:17.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:16 smithi179 ceph-mon[35781]: pgmap v414: 129 pgs: 129 active+clean; 748 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:43:19.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:18 smithi038 ceph-mon[31227]: pgmap v415: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:43:19.298 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:18 smithi047 ceph-mon[35327]: pgmap v415: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:43:19.299 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:18 smithi179 ceph-mon[35781]: pgmap v415: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:43:21.028 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:20 smithi047 ceph-mon[35327]: pgmap v416: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:43:21.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:20 smithi038 ceph-mon[31227]: pgmap v416: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:43:21.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:20 smithi179 ceph-mon[35781]: pgmap v416: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:43:23.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:22 smithi038 ceph-mon[31227]: pgmap v417: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:43:23.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:22 smithi179 ceph-mon[35781]: pgmap v417: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:43:23.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:22 smithi047 ceph-mon[35327]: pgmap v417: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:43:25.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:25 smithi038 ceph-mon[31227]: pgmap v418: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:25 smithi179 ceph-mon[35781]: pgmap v418: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:25.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:25 smithi047 ceph-mon[35327]: pgmap v418: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:43:27.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:27 smithi038 ceph-mon[31227]: pgmap v419: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:27 smithi179 ceph-mon[35781]: pgmap v419: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:27.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:27 smithi047 ceph-mon[35327]: pgmap v419: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:43:27.793 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:43:28.068 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:28.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 -- 172.21.15.38:0/524157641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 msgr2=0x7f0dac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/524157641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac1067e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009900 tx=0x7f0d9c02a410 comp rx=0 tx=0).stop 2024-09-19T05:43:28.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 -- 172.21.15.38:0/524157641 shutdown_connections 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/524157641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac106f10 0x7f0dac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/524157641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac1067e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/524157641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac101de0 0x7f0dac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.573+0000 7f0db3ed8640 1 -- 172.21.15.38:0/524157641 >> 172.21.15.38:0/524157641 conn(0x7f0dac0fd8c0 msgr2=0x7f0dac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- 172.21.15.38:0/524157641 shutdown_connections 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- 172.21.15.38:0/524157641 wait complete. 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 Processor -- start 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- start start 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac101de0 0x7f0dac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac106f10 0x7f0dac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac111f70 con 0x7f0dac102790 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac111df0 con 0x7f0dac106f10 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db3ed8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1120f0 con 0x7f0dac101de0 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db1473640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db1473640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46592/0 (socket says 172.21.15.38:46592) 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.574+0000 7f0db1473640 1 -- 172.21.15.38:0/3501083506 learned_addr learned my addr 172.21.15.38:0/3501083506 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db2475640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac106f10 0x7f0dac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1c74640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac101de0 0x7f0dac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 -- 172.21.15.38:0/3501083506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac101de0 msgr2=0x7f0dac198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac101de0 0x7f0dac198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 -- 172.21.15.38:0/3501083506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac106f10 msgr2=0x7f0dac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac106f10 0x7f0dac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 -- 172.21.15.38:0/3501083506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19d3c0 con 0x7f0dac102790 2024-09-19T05:43:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db1473640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006010 tx=0x7f0d9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c03f030 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9c003ce0 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c035070 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19d650 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.575+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac19dae0 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.576+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac1021c0 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.578+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0d9c042030 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.578+0000 7f0da2ffd640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 0x7f0d980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.578+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f0d9c0f0490 con 0x7f0dac102790 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.578+0000 7f0db1c74640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 0x7f0d980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.579+0000 7f0db1c74640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 0x7f0d980a5700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009b60 tx=0x7f0da8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.582+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d9c0bceb0 con 0x7f0dac102790 2024-09-19T05:43:28.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.792+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dac02d7c0 con 0x7f0dac102790 2024-09-19T05:43:28.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.793+0000 7f0da2ffd640 1 -- 172.21.15.38:0/3501083506 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f0d9c0bc7b0 con 0x7f0dac102790 2024-09-19T05:43:28.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:43:28.795 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:43:28.795 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:43:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 msgr2=0x7f0d980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 0x7f0d980a5700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009b60 tx=0x7f0da8005f90 comp rx=0 tx=0).stop 2024-09-19T05:43:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 msgr2=0x7f0dac198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006010 tx=0x7f0d9c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 shutdown_connections 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f0d980a3240 0x7f0d980a5700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dac106f10 0x7f0dac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0dac102790 0x7f0dac198950 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 --2- 172.21.15.38:0/3501083506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0dac101de0 0x7f0dac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.795+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 >> 172.21.15.38:0/3501083506 conn(0x7f0dac0fd8c0 msgr2=0x7f0dac0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.796+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 shutdown_connections 2024-09-19T05:43:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:28.796+0000 7f0db3ed8640 1 -- 172.21.15.38:0/3501083506 wait complete. 2024-09-19T05:43:29.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:29 smithi038 ceph-mon[31227]: pgmap v420: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:29.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:29 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3501083506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:29.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:29 smithi179 ceph-mon[35781]: pgmap v420: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:29.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:29 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3501083506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:29.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:29 smithi047 ceph-mon[35327]: pgmap v420: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:43:29.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:29 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3501083506' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:43:29.578 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 52e7bead --release --await' 2024-09-19T05:43:29.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 52e7bead --release --await 2024-09-19T05:43:29.838 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 -- 172.21.15.38:0/3762790948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 msgr2=0x7f29301031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 --2- 172.21.15.38:0/3762790948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f29301031c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292002a480 comp rx=0 tx=0).stop 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 -- 172.21.15.38:0/3762790948 shutdown_connections 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 --2- 172.21.15.38:0/3762790948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f293010c180 0x7f293010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 --2- 172.21.15.38:0/3762790948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2930103700 0x7f293010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 --2- 172.21.15.38:0/3762790948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f29301031c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 -- 172.21.15.38:0/3762790948 >> 172.21.15.38:0/3762790948 conn(0x7f29300fc7a0 msgr2=0x7f29300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 -- 172.21.15.38:0/3762790948 shutdown_connections 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.338+0000 7f2934a33640 1 -- 172.21.15.38:0/3762790948 wait complete. 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.339+0000 7f2934a33640 1 Processor -- start 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.339+0000 7f2934a33640 1 -- start start 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2930103700 0x7f2930198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293010c180 0x7f293019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2930112000 con 0x7f2930102de0 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2930111e80 con 0x7f2930103700 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2934a33640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2930112180 con 0x7f293010c180 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46610/0 (socket says 172.21.15.38:46610) 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 -- 172.21.15.38:0/3422387200 learned_addr learned my addr 172.21.15.38:0/3422387200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f2925d9b640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2930103700 0x7f2930198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293010c180 0x7f293019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 -- 172.21.15.38:0/3422387200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293010c180 msgr2=0x7f293019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293010c180 0x7f293019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 -- 172.21.15.38:0/3422387200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2930103700 msgr2=0x7f2930198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2930103700 0x7f2930198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 -- 172.21.15.38:0/3422387200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293019d530 con 0x7f2930102de0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.340+0000 7f292e59c640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2920006010 tx=0x7f2920003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.341+0000 7f2927fff640 1 -- 172.21.15.38:0/3422387200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292003f070 con 0x7f2930102de0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.341+0000 7f2927fff640 1 -- 172.21.15.38:0/3422387200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920031c90 con 0x7f2930102de0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.341+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293019d7c0 con 0x7f2930102de0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.341+0000 7f2927fff640 1 -- 172.21.15.38:0/3422387200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920035030 con 0x7f2930102de0 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.341+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a5100 con 0x7f2930102de0 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.343+0000 7f2927fff640 1 -- 172.21.15.38:0/3422387200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2920042070 con 0x7f2930102de0 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.343+0000 7f2936e0c740 1 Processor -- start 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 -- start start 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f831d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 0x558b8f834630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558b8f78c620 con 0x558b8f827060 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558b8f733e50 con 0x558b8f827980 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2936e0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b8f7885e0 con 0x558b8f832240 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2927fff640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 0x7f291c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f831d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46620/0 (socket says 172.21.15.38:46620) 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 -- 172.21.15.38:0/3139956627 learned_addr learned my addr 172.21.15.38:0/3139956627 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2925d9b640 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 0x558b8f834630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 0x7f291c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f2927fff640 1 -- 172.21.15.38:0/3422387200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f29200f0540 con 0x7f2930102de0 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 -- 172.21.15.38:0/3139956627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 msgr2=0x558b8f834630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.344+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 0x558b8f834630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f292ed9d640 1 -- 172.21.15.38:0/3139956627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 msgr2=0x558b8f831d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f831d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f292ed9d640 1 -- 172.21.15.38:0/3139956627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8f724d40 con 0x558b8f827060 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f2925d9b640 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 0x558b8f834630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2928005fd0 tx=0x7f29280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b635bfa62ff07b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f292ed9d640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 0x7f291c0a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2928011cb0 tx=0x7f2928016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f2924d99640 1 -- 172.21.15.38:0/3139956627 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292800ad10 con 0x558b8f827060 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f2924d99640 1 -- 172.21.15.38:0/3139956627 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928003e50 con 0x558b8f827060 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.345+0000 7f2924d99640 1 -- 172.21.15.38:0/3139956627 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928004530 con 0x558b8f827060 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- 172.21.15.38:0/3139956627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 msgr2=0x558b8f827440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2928005fd0 tx=0x7f29280117b0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- 172.21.15.38:0/3139956627 shutdown_connections 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f832240 0x558b8f834630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f831d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 --2- 172.21.15.38:0/3139956627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f827060 0x558b8f827440 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- 172.21.15.38:0/3139956627 >> 172.21.15.38:0/3139956627 conn(0x558b8f8235f0 msgr2=0x558b8f8239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- 172.21.15.38:0/3139956627 shutdown_connections 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- 172.21.15.38:0/3139956627 wait complete. 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 -- start start 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.346+0000 7f2936e0c740 1 Processor -- start 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f82f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f82f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x558b8f839220 con 0x558b8f832240 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558b8f714f10 con 0x558b8f827980 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b8f8393a0 con 0x558b8f827060 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2936e0c740 10 client.0 _ll_register_callbacks cb 0x558b8f738db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46630/0 (socket says 172.21.15.38:46630) 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 -- 172.21.15.38:0/1933558778 learned_addr learned my addr 172.21.15.38:0/1933558778 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f2925d9b640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f82f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292e59c640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f82f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 -- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 msgr2=0x558b8f82f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f82f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 -- 172.21.15.38:0/1933558778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 msgr2=0x558b8f82f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f82f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292ed9d640 1 -- 172.21.15.38:0/1933558778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8f49b580 con 0x558b8f832240 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.347+0000 7f292e59c640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f82f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f292ed9d640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2928005f80 tx=0x7f292800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637f38045c71eb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928030070 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f2928003b80 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2928003e50 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f292803c050 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b8f565960 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928003e50 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f292801d070 con 0x558b8f832240 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 10 client.15570 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 20 client.15570 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f846190 0x558b8f848580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 4 client.15570 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.348+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8f799b90 con 0x558b8f846190 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.349+0000 7f292e59c640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f846190 0x558b8f848580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.349+0000 7f292e59c640 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f846190 0x558b8f848580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66dcc90b36b46fef server_cookie=850629aa1b1c5d22 in_seq=0 out_seq=0 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.349+0000 7f2915ffb640 10 client.15570 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2915ffb640 1 -- 172.21.15.38:0/1933558778 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b8f799b90 con 0x558b8f846190 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2915ffb640 10 client.15570 handle_command_reply: tid=0 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 client.15570 shutdown 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 msgr2=0x558b8f82fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2928005f80 tx=0x7f292800ced0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 shutdown_connections 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f846190 0x558b8f848580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f82fc40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f82f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2915ffb640 0 client.15570 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 --2- 172.21.15.38:0/1933558778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f82f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 >> 172.21.15.38:0/1933558778 conn(0x558b8f8235f0 msgr2=0x558b8f829320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 shutdown_connections 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 1 -- 172.21.15.38:0/1933558778 wait complete. 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.350+0000 7f2936e0c740 20 client.15570 trim_cache size 0 max 16384 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.367+0000 7f2936e0c740 1 Processor -- start 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 -- start start 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 0x558b8f831090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827980 0x558b8f829ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29301acbb0 con 0x558b8f832240 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29301ae6f0 con 0x558b8f827060 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2936e0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29301ad280 con 0x558b8f827980 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 0x558b8f831090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46634/0 (socket says 172.21.15.38:46634) 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 -- 172.21.15.38:0/2361469424 learned_addr learned my addr 172.21.15.38:0/2361469424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f2925d9b640 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827980 0x558b8f829ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 -- 172.21.15.38:0/2361469424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827980 msgr2=0x558b8f829ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827980 0x558b8f829ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 -- 172.21.15.38:0/2361469424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 msgr2=0x558b8f831090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 0x558b8f831090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 -- 172.21.15.38:0/2361469424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8f8246a0 con 0x558b8f832240 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292e59c640 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 0x558b8f831090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.368+0000 7f292ed9d640 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f292800acb0 tx=0x7f292800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad097656dc4d78d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2917fff640 1 -- 172.21.15.38:0/2361469424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928030070 con 0x558b8f832240 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2917fff640 1 -- 172.21.15.38:0/2361469424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928021480 con 0x558b8f832240 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2917fff640 1 -- 172.21.15.38:0/2361469424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280218a0 con 0x558b8f832240 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- 172.21.15.38:0/2361469424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 msgr2=0x558b8f84dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f292800acb0 tx=0x7f292800a4d0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- 172.21.15.38:0/2361469424 shutdown_connections 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f84dc30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827980 0x558b8f829ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 --2- 172.21.15.38:0/2361469424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827060 0x558b8f831090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- 172.21.15.38:0/2361469424 >> 172.21.15.38:0/2361469424 conn(0x558b8f8235f0 msgr2=0x558b8f822750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- 172.21.15.38:0/2361469424 shutdown_connections 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- 172.21.15.38:0/2361469424 wait complete. 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.369+0000 7f2936e0c740 1 -- start start 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 Processor -- start 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f83e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f83ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f29301ae020 con 0x558b8f832240 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29301ad950 con 0x558b8f827980 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29301aab90 con 0x558b8f827060 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2936e0c740 10 client.0 _ll_register_callbacks cb 0x7f292800aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46650/0 (socket says 172.21.15.38:46650) 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 -- 172.21.15.38:0/996494447 learned_addr learned my addr 172.21.15.38:0/996494447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292e59c640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f83e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2925d9b640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f83ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 -- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 msgr2=0x558b8f83e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f83e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 -- 172.21.15.38:0/996494447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 msgr2=0x558b8f83ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f83ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292ed9d640 1 -- 172.21.15.38:0/996494447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8f83bdf0 con 0x558b8f832240 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f292e59c640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f83e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.370+0000 7f2925d9b640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f83ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f292ed9d640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f292800aa90 tx=0x7f2928021ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e334b2d7a4d6ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928045030 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f2928026070 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2928021480 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2928052050 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 10 client.15576 fetch_fsmap learned FSMap version 28 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b8f8b1690 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928021480 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f292801d040 con 0x558b8f832240 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 10 client.15576 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 20 client.15576 populate_metadata read hostname 'smithi038' 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f8b1cd0 0x558b8f8b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 4 client.15576 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "52e7bead", "release": true, "await": true}] 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.371+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "52e7bead", "release": true, "await": true}) -- 0x558b8f7940a0 con 0x558b8f8b1cd0 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.372+0000 7f292e59c640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f8b1cd0 0x558b8f8b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.372+0000 7f292e59c640 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f8b1cd0 0x558b8f8b40f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66895fadefad63b8 server_cookie=a5c3e7c0aa3b365e in_seq=0 out_seq=0 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.372+0000 7f2915ffb640 10 client.15576 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2915ffb640 1 -- 172.21.15.38:0/996494447 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558b8f7940a0 con 0x558b8f8b1cd0 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2915ffb640 10 client.15576 handle_command_reply: tid=0 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 client.15576 shutdown 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 msgr2=0x558b8f83b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f292800aa90 tx=0x7f2928021ad0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 shutdown_connections 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x558b8f8b1cd0 0x558b8f8b40f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x558b8f832240 0x558b8f83b5c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2915ffb640 0 client.15576 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b8f827980 0x558b8f83ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 --2- 172.21.15.38:0/996494447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b8f827060 0x558b8f83e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 >> 172.21.15.38:0/996494447 conn(0x558b8f8235f0 msgr2=0x558b8f82efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 shutdown_connections 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 1 -- 172.21.15.38:0/996494447 wait complete. 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2936e0c740 20 client.15576 trim_cache size 0 max 16384 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 12, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "52e7bead": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 12, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 140.3, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 msgr2=0x7f291c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 0x7f291c0a5700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2928011cb0 tx=0x7f2928016000 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.376+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 msgr2=0x7f2930198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2920006010 tx=0x7f2920003ba0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 shutdown_connections 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f291c0a3240 0x7f291c0a5700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f293010c180 0x7f293019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2930103700 0x7f2930198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 --2- 172.21.15.38:0/3422387200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2930102de0 0x7f2930198580 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 >> 172.21.15.38:0/3422387200 conn(0x7f29300fc7a0 msgr2=0x7f29300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 shutdown_connections 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:43:30.377+0000 7f2934a33640 1 -- 172.21.15.38:0/3422387200 wait complete. 2024-09-19T05:43:31.127 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '52e7bead', total seconds elapsed: 23.7 2024-09-19T05:43:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:31 smithi038 ceph-mon[31227]: pgmap v421: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:43:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:31 smithi179 ceph-mon[35781]: pgmap v421: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:43:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:31 smithi047 ceph-mon[35327]: pgmap v421: 129 pgs: 129 active+clean; 754 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:43:33.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:33 smithi038 ceph-mon[31227]: pgmap v422: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:43:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:33 smithi179 ceph-mon[35781]: pgmap v422: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:43:33.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:33 smithi047 ceph-mon[35327]: pgmap v422: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:43:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:35 smithi179 ceph-mon[35781]: pgmap v423: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:35.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:35 smithi047 ceph-mon[35327]: pgmap v423: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:35 smithi038 ceph-mon[31227]: pgmap v423: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:37 smithi179 ceph-mon[35781]: pgmap v424: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:37.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:37 smithi047 ceph-mon[35327]: pgmap v424: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:37 smithi038 ceph-mon[31227]: pgmap v424: 129 pgs: 129 active+clean; 776 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:43:39.446 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:39 smithi038 ceph-mon[31227]: pgmap v425: 129 pgs: 129 active+clean; 833 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:43:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:39 smithi179 ceph-mon[35781]: pgmap v425: 129 pgs: 129 active+clean; 833 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:43:39.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:39 smithi047 ceph-mon[35327]: pgmap v425: 129 pgs: 129 active+clean; 833 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:43:41.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:41 smithi179 ceph-mon[35781]: pgmap v426: 129 pgs: 129 active+clean; 833 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:43:41.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:41 smithi047 ceph-mon[35327]: pgmap v426: 129 pgs: 129 active+clean; 833 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:43:41.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:41 smithi038 ceph-mon[31227]: pgmap v426: 129 pgs: 129 active+clean; 833 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:43:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:43 smithi179 ceph-mon[35781]: pgmap v427: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:43:43.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:43 smithi047 ceph-mon[35327]: pgmap v427: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:43:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:43 smithi038 ceph-mon[31227]: pgmap v427: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:43:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:45 smithi179 ceph-mon[35781]: pgmap v428: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:45 smithi047 ceph-mon[35327]: pgmap v428: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:45.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:45 smithi038 ceph-mon[31227]: pgmap v428: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:47 smithi179 ceph-mon[35781]: pgmap v429: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:43:47.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:47 smithi047 ceph-mon[35327]: pgmap v429: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:43:47.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:47 smithi038 ceph-mon[31227]: pgmap v429: 129 pgs: 129 active+clean; 886 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:43:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:49 smithi179 ceph-mon[35781]: pgmap v430: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:49 smithi047 ceph-mon[35327]: pgmap v430: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:49 smithi038 ceph-mon[31227]: pgmap v430: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:51.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:51 smithi047 ceph-mon[35327]: pgmap v431: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:51 smithi038 ceph-mon[31227]: pgmap v431: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:51 smithi179 ceph-mon[35781]: pgmap v431: 129 pgs: 129 active+clean; 944 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:43:53.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:53 smithi047 ceph-mon[35327]: pgmap v432: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:53 smithi038 ceph-mon[31227]: pgmap v432: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:53 smithi179 ceph-mon[35781]: pgmap v432: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:43:55.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:55 smithi047 ceph-mon[35327]: pgmap v433: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:43:55.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:55 smithi179 ceph-mon[35781]: pgmap v433: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:43:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:55 smithi038 ceph-mon[31227]: pgmap v433: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:43:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:57 smithi047 ceph-mon[35327]: pgmap v434: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:43:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:57 smithi038 ceph-mon[31227]: pgmap v434: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:43:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:57 smithi179 ceph-mon[35781]: pgmap v434: 129 pgs: 129 active+clean; 1004 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:43:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:43:59 smithi047 ceph-mon[35327]: pgmap v435: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:43:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:43:59 smithi038 ceph-mon[31227]: pgmap v435: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:43:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:43:59 smithi179 ceph-mon[35781]: pgmap v435: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:44:01.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:01 smithi047 ceph-mon[35327]: pgmap v436: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:01.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:01 smithi038 ceph-mon[31227]: pgmap v436: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:01 smithi179 ceph-mon[35781]: pgmap v436: 129 pgs: 129 active+clean; 1.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:03 smithi038 ceph-mon[31227]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:03 smithi179 ceph-mon[35781]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:03 smithi047 ceph-mon[35327]: pgmap v437: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:05 smithi038 ceph-mon[31227]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:05 smithi179 ceph-mon[35781]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:05.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:05 smithi047 ceph-mon[35327]: pgmap v438: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:44:06.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:06 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:44:06.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:06 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:44:06.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:06 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:44:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:07 smithi038 ceph-mon[31227]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:44:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:44:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:44:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:44:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:07 smithi179 ceph-mon[35781]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:44:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:44:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:44:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:44:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:07 smithi047 ceph-mon[35327]: pgmap v439: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:44:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:44:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:44:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:44:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:09 smithi038 ceph-mon[31227]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:09 smithi179 ceph-mon[35781]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:09.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:09 smithi047 ceph-mon[35327]: pgmap v440: 129 pgs: 129 active+clean; 1.1 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:11.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:11 smithi038 ceph-mon[31227]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:11 smithi179 ceph-mon[35781]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:11 smithi047 ceph-mon[35327]: pgmap v441: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:13 smithi038 ceph-mon[31227]: pgmap v442: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:13 smithi179 ceph-mon[35781]: pgmap v442: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:13.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:13 smithi047 ceph-mon[35327]: pgmap v442: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:15.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:15 smithi038 ceph-mon[31227]: pgmap v443: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:15 smithi179 ceph-mon[35781]: pgmap v443: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:15 smithi047 ceph-mon[35327]: pgmap v443: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:17.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:17 smithi038 ceph-mon[31227]: pgmap v444: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:17 smithi179 ceph-mon[35781]: pgmap v444: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:17.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:17 smithi047 ceph-mon[35327]: pgmap v444: 129 pgs: 129 active+clean; 1.2 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:19.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:19 smithi038 ceph-mon[31227]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:19 smithi179 ceph-mon[35781]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:19.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:19 smithi047 ceph-mon[35327]: pgmap v445: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:21.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:21 smithi038 ceph-mon[31227]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:21 smithi179 ceph-mon[35781]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:21 smithi047 ceph-mon[35327]: pgmap v446: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:23.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:23 smithi038 ceph-mon[31227]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:23 smithi179 ceph-mon[35781]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:23.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:23 smithi047 ceph-mon[35327]: pgmap v447: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:25.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:25 smithi038 ceph-mon[31227]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:25 smithi179 ceph-mon[35781]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:25 smithi047 ceph-mon[35327]: pgmap v448: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:1/600: open fd = 3 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:2/600: open fd = 3 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:3/600: open fd = 3 2024-09-19T05:44:26.787 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:4/600: open fd = 3 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:5/600: open fd = 3 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:6/600: open fd = 3 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.788 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:7/600: open fd = 3 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:8/600: open fd = 3 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:9/600: open fd = 3 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.789 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:10/600: open fd = 3 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:11/600: open fd = 3 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:12/600: open fd = 3 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.790 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:13/600: open fd = 3 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:14/600: open fd = 3 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:15/600: open fd = 3 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.791 INFO:tasks.workunit.client.0.smithi038.stdout:16/600: open fd = 3 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:17/600: open fd = 3 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:18/600: open fd = 3 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:19/600: open fd = 3 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.792 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:20/600: open fd = 3 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:21/600: open fd = 3 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:22/600: open fd = 3 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.793 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:23/600: open fd = 3 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:24/600: open fd = 3 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:25/600: open fd = 3 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.794 INFO:tasks.workunit.client.0.smithi038.stdout:26/600: open fd = 3 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:27/600: open fd = 3 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:28/600: open fd = 3 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:29/600: open fd = 3 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:30/600: open fd = 3 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:31/600: open fd = 3 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:32/600: open fd = 3 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.796 INFO:tasks.workunit.client.0.smithi038.stdout:33/600: open fd = 3 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:34/600: open fd = 3 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:35/600: open fd = 3 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:36/600: open fd = 3 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:37/600: open fd = 3 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:38/600: open fd = 3 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:39/600: open fd = 3 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.798 INFO:tasks.workunit.client.0.smithi038.stdout:40/600: open fd = 3 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:41/600: open fd = 3 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:42/600: open fd = 3 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:43/600: open fd = 3 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:44/600: open fd = 3 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:45/600: open fd = 3 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:46/600: open fd = 3 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.800 INFO:tasks.workunit.client.0.smithi038.stdout:47/600: open fd = 3 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:48/600: open fd = 3 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:49/600: open fd = 3 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:50/600: open fd = 3 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:51/600: open fd = 3 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:52/600: open fd = 3 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:53/600: open fd = 3 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:54/600: open fd = 3 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:55/600: open fd = 3 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:56/600: open fd = 3 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:57/600: open fd = 3 2024-09-19T05:44:26.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:58/600: open fd = 3 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:59/600: open fd = 3 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:60/600: open fd = 3 2024-09-19T05:44:26.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:61/600: open fd = 3 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:62/600: open fd = 3 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:63/600: open fd = 3 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:44:26.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:44:27.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:27 smithi038 ceph-mon[31227]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:27.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:27 smithi179 ceph-mon[35781]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:27.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:27 smithi047 ceph-mon[35327]: pgmap v449: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:29.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:29 smithi038 ceph-mon[31227]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:44:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:29 smithi179 ceph-mon[35781]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:44:29.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:29 smithi047 ceph-mon[35327]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:44:31.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:31 smithi038 ceph-mon[31227]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:31.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:31 smithi179 ceph-mon[35781]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:31.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:31 smithi047 ceph-mon[35327]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:33 smithi038 ceph-mon[31227]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:33 smithi179 ceph-mon[35781]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:33.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:33 smithi047 ceph-mon[35327]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:35 smithi038 ceph-mon[31227]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:35 smithi179 ceph-mon[35781]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:35 smithi047 ceph-mon[35327]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:37 smithi038 ceph-mon[31227]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:37 smithi179 ceph-mon[35781]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:37.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:37 smithi047 ceph-mon[35327]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:39 smithi038 ceph-mon[31227]: pgmap v455: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:39 smithi179 ceph-mon[35781]: pgmap v455: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:39.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:39 smithi047 ceph-mon[35327]: pgmap v455: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:44:41.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:41 smithi038 ceph-mon[31227]: pgmap v456: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:41 smithi179 ceph-mon[35781]: pgmap v456: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:41.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:41 smithi047 ceph-mon[35327]: pgmap v456: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:43 smithi038 ceph-mon[31227]: pgmap v457: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:43 smithi179 ceph-mon[35781]: pgmap v457: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:43.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:43 smithi047 ceph-mon[35327]: pgmap v457: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-19T05:44:45.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:45 smithi038 ceph-mon[31227]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:45 smithi179 ceph-mon[35781]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:45.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:45 smithi047 ceph-mon[35327]: pgmap v458: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:46.715 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T05:44:46.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:44:46.987 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:44:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 -- 172.21.15.38:0/958180305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 msgr2=0x7f4a8410ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 --2- 172.21.15.38:0/958180305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8410ead0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009cd0 tx=0x7f4a7c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 -- 172.21.15.38:0/958180305 shutdown_connections 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 --2- 172.21.15.38:0/958180305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8410ead0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 --2- 172.21.15.38:0/958180305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100ce0 0x7f4a84106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 --2- 172.21.15.38:0/958180305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100330 0x7f4a84100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 -- 172.21.15.38:0/958180305 >> 172.21.15.38:0/958180305 conn(0x7f4a840fbee0 msgr2=0x7f4a840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 -- 172.21.15.38:0/958180305 shutdown_connections 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.503+0000 7f4a8962a640 1 -- 172.21.15.38:0/958180305 wait complete. 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 Processor -- start 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 -- start start 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100330 0x7f4a84198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100ce0 0x7f4a84198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84111f80 con 0x7f4a84106f30 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84111e00 con 0x7f4a84100330 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a8962a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84112100 con 0x7f4a84100ce0 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.504+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a827fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100ce0 0x7f4a84198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53304/0 (socket says 172.21.15.38:53304) 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 -- 172.21.15.38:0/1747287704 learned_addr learned my addr 172.21.15.38:0/1747287704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a82ffd640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100330 0x7f4a84198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 -- 172.21.15.38:0/1747287704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100ce0 msgr2=0x7f4a84198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100ce0 0x7f4a84198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 -- 172.21.15.38:0/1747287704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100330 msgr2=0x7f4a84198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100330 0x7f4a84198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 -- 172.21.15.38:0/1747287704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8419d4d0 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a837fe640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005cc0 tx=0x7f4a7c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c004040 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c0046c0 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8419d760 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c046480 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.505+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841a50a0 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.506+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84100710 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.508+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4a7c0305a0 con 0x7f4a84106f30 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.508+0000 7f4a6bfff640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 0x7f4a6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.508+0000 7f4a82ffd640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 0x7f4a6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.508+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4a7c02e030 con 0x7f4a84106f30 2024-09-19T05:44:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.509+0000 7f4a82ffd640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 0x7f4a6c0a5700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400bbe0 tx=0x7f4a7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.511+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a7c04b060 con 0x7f4a84106f30 2024-09-19T05:44:47.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:47 smithi038 ceph-mon[31227]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:47.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:47 smithi179 ceph-mon[35781]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:47.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:47 smithi047 ceph-mon[35327]: pgmap v459: 129 pgs: 129 active+clean; 1.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:44:47.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.729+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a8410a100 con 0x7f4a84106f30 2024-09-19T05:44:47.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.729+0000 7f4a6bfff640 1 -- 172.21.15.38:0/1747287704 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f4a7c0bdf30 con 0x7f4a84106f30 2024-09-19T05:44:47.731 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:44:47.732 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:44:47.732 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:44:47.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 msgr2=0x7f4a6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 0x7f4a6c0a5700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400bbe0 tx=0x7f4a7400b390 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 msgr2=0x7f4a8419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005cc0 tx=0x7f4a7c003970 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 shutdown_connections 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4a6c0a3240 0x7f4a6c0a5700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a84106f30 0x7f4a8419cdf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a84100ce0 0x7f4a84198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 --2- 172.21.15.38:0/1747287704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a84100330 0x7f4a84198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.731+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 >> 172.21.15.38:0/1747287704 conn(0x7f4a840fbee0 msgr2=0x7f4a840fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.732+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 shutdown_connections 2024-09-19T05:44:47.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:47.732+0000 7f4a8962a640 1 -- 172.21.15.38:0/1747287704 wait complete. 2024-09-19T05:44:48.492 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T05:44:48.492 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T05:44:48.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:48 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1747287704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:44:48.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:48 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1747287704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:44:48.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:48 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1747287704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:44:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- 172.21.15.38:0/1963979436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 msgr2=0x7f3ab810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 --2- 172.21.15.38:0/1963979436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab810eaa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009ca0 tx=0x7f3ab002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- 172.21.15.38:0/1963979436 shutdown_connections 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 --2- 172.21.15.38:0/1963979436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab810eaa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 --2- 172.21.15.38:0/1963979436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 --2- 172.21.15.38:0/1963979436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- 172.21.15.38:0/1963979436 >> 172.21.15.38:0/1963979436 conn(0x7f3ab80fd8b0 msgr2=0x7f3ab80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- 172.21.15.38:0/1963979436 shutdown_connections 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- 172.21.15.38:0/1963979436 wait complete. 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 Processor -- start 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.258+0000 7f3abe45f640 1 -- start start 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8111f80 con 0x7f3ab8106f20 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8111e00 con 0x7f3ab8101df0 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abe45f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112100 con 0x7f3ab81027a0 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53330/0 (socket says 172.21.15.38:53330) 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 -- 172.21.15.38:0/3136446351 learned_addr learned my addr 172.21.15.38:0/3136446351 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 -- 172.21.15.38:0/3136446351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 msgr2=0x7f3ab8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 -- 172.21.15.38:0/3136446351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 msgr2=0x7f3ab80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3abc9fc640 1 -- 172.21.15.38:0/3136446351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a6800 con 0x7f3ab8106f20 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.259+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3abc9fc640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0031a20 tx=0x7f3ab0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3ab57fa640 1 -- 172.21.15.38:0/3136446351 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0003b70 con 0x7f3ab8106f20 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3ab57fa640 1 -- 172.21.15.38:0/3136446351 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab0003d10 con 0x7f3ab8106f20 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a6a90 con 0x7f3ab8106f20 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3ab57fa640 1 -- 172.21.15.38:0/3136446351 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab002fc80 con 0x7f3ab8106f20 2024-09-19T05:44:49.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.260+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a6fd0 con 0x7f3ab8106f20 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.262+0000 7f3ab57fa640 1 -- 172.21.15.38:0/3136446351 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3ab00046a0 con 0x7f3ab8106f20 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.262+0000 7f3ac0838740 1 Processor -- start 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 -- start start 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 0x5649973a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 0x5649973afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5649973082a0 con 0x5649973a2760 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5649972afb10 con 0x5649973ad940 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ac0838740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564997304260 con 0x5649973a3080 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 0x5649973a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 0x5649973a2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53334/0 (socket says 172.21.15.38:53334) 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3943894208 learned_addr learned my addr 172.21.15.38:0/3943894208 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab57fa640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 0x7f3aa00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 0x5649973afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3abc9fc640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 0x7f3aa00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3943894208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 msgr2=0x5649973ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3943894208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 msgr2=0x5649973a2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab57fa640 1 -- 172.21.15.38:0/3136446351 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f3ab00f0ea0 con 0x7f3ab8106f20 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.263+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 0x5649973a2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.264+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3943894208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564997079b90 con 0x5649973ad940 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.264+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 0x5649973afd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3aac012b30 tx=0x7f3aac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8f9a0b715a30aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.264+0000 7f3a9e7fc640 1 -- 172.21.15.38:0/3943894208 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00d800 con 0x5649973ad940 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.264+0000 7f3a9e7fc640 1 -- 172.21.15.38:0/3943894208 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac00de80 con 0x5649973ad940 2024-09-19T05:44:49.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.264+0000 7f3a9e7fc640 1 -- 172.21.15.38:0/3943894208 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac01e3c0 con 0x5649973ad940 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- 172.21.15.38:0/3943894208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 msgr2=0x5649973afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3abc9fc640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 0x7f3aa00a5700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00b95f0 tx=0x7f3ab00b7bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 0x5649973afd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3aac012b30 tx=0x7f3aac004730 comp rx=0 tx=0).stop 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- 172.21.15.38:0/3943894208 shutdown_connections 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973ad940 0x5649973afd30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3943894208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a2760 0x5649973a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- 172.21.15.38:0/3943894208 >> 172.21.15.38:0/3943894208 conn(0x56499739ecf0 msgr2=0x56499739f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- 172.21.15.38:0/3943894208 shutdown_connections 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- 172.21.15.38:0/3943894208 wait complete. 2024-09-19T05:44:49.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 -- start start 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.265+0000 7f3ac0838740 1 Processor -- start 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5649973b0510 con 0x5649973ad940 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5649972906e0 con 0x5649973a2760 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5649973b0690 con 0x5649973a3080 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ac0838740 10 client.0 _ll_register_callbacks cb 0x5649972b4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53342/0 (socket says 172.21.15.38:53342) 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3541802343 learned_addr learned my addr 172.21.15.38:0/3541802343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 msgr2=0x5649973aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3541802343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 msgr2=0x5649973aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 -- 172.21.15.38:0/3541802343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649973b99b0 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0059a0 tx=0x7f3aac00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30f29a4e4b4f217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.266+0000 7f3a977fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004030 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f3aac0041d0 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649973159d0 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aac032050 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 10 client.15594 fetch_fsmap learned FSMap version 28 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649973bc340 con 0x5649973ad940 2024-09-19T05:44:49.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5649973159d0 con 0x5649973ad940 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3aac01f410 con 0x5649973ad940 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 10 client.15594 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 20 client.15594 populate_metadata read hostname 'smithi038' 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x5649973bfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 4 client.15594 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.267+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499730fd20 con 0x5649973bd690 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.268+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x5649973bfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.268+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x5649973bfa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7dd306ab5d08db server_cookie=3d7fc750a9eed2cc in_seq=0 out_seq=0 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.268+0000 7f3a977fe640 10 client.15594 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3a977fe640 1 -- 172.21.15.38:0/3541802343 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56499730fd20 con 0x5649973bd690 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3a977fe640 10 client.15594 handle_command_reply: tid=0 2024-09-19T05:44:49.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 client.15594 shutdown 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 msgr2=0x5649973ab230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0059a0 tx=0x7f3aac00fb10 comp rx=0 tx=0).stop 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 shutdown_connections 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x5649973bfa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973ab230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3a977fe640 0 client.15594 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 --2- 172.21.15.38:0/3541802343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 >> 172.21.15.38:0/3541802343 conn(0x56499739ecf0 msgr2=0x5649973a4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 shutdown_connections 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 1 -- 172.21.15.38:0/3541802343 wait complete. 2024-09-19T05:44:49.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.269+0000 7f3ac0838740 20 client.15594 trim_cache size 0 max 16384 2024-09-19T05:44:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.286+0000 7f3ac0838740 1 Processor -- start 2024-09-19T05:44:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 -- start start 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0ec0 con 0x5649973ad940 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b2a00 con 0x5649973a2760 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ac0838740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b1590 con 0x5649973a3080 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53352/0 (socket says 172.21.15.38:53352) 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 -- 172.21.15.38:0/524847870 learned_addr learned my addr 172.21.15.38:0/524847870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:49.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 -- 172.21.15.38:0/524847870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 msgr2=0x5649973c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 -- 172.21.15.38:0/524847870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 msgr2=0x5649973a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 -- 172.21.15.38:0/524847870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499739eb20 con 0x5649973ad940 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00de50 tx=0x7f3aac0124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb8cb833d903229 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.287+0000 7f3a9dffb640 1 -- 172.21.15.38:0/524847870 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac016070 con 0x5649973ad940 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3a9dffb640 1 -- 172.21.15.38:0/524847870 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac005bb0 con 0x5649973ad940 2024-09-19T05:44:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3a9dffb640 1 -- 172.21.15.38:0/524847870 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00fb80 con 0x5649973ad940 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 -- 172.21.15.38:0/524847870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 msgr2=0x5649973c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00de50 tx=0x7f3aac0124f0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 -- 172.21.15.38:0/524847870 shutdown_connections 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973ad940 0x5649973c5820 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973a3080 0x5649973c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 --2- 172.21.15.38:0/524847870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 -- 172.21.15.38:0/524847870 >> 172.21.15.38:0/524847870 conn(0x56499739ecf0 msgr2=0x5649973ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 -- 172.21.15.38:0/524847870 shutdown_connections 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.288+0000 7f3ac0838740 1 -- 172.21.15.38:0/524847870 wait complete. 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 -- start start 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 Processor -- start 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973ad940 0x5649974272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b2330 con 0x5649973a3080 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b1c60 con 0x5649973a2760 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81aeea0 con 0x5649973ad940 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ac0838740 10 client.0 _ll_register_callbacks cb 0x7f3aac00f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53358/0 (socket says 172.21.15.38:53358) 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ab7fff640 1 -- 172.21.15.38:0/1343228815 learned_addr learned my addr 172.21.15.38:0/1343228815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.289+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973ad940 0x5649974272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 -- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973ad940 msgr2=0x5649974272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973ad940 0x5649974272b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 -- 172.21.15.38:0/1343228815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 msgr2=0x5649973b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 -- 172.21.15.38:0/1343228815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499739f980 con 0x5649973a3080 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ab7fff640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8004700 tx=0x7f3aa8012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d89c31c08aa1a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3a977fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8023070 con 0x5649973a3080 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f3aa8005020 con 0x5649973a3080 2024-09-19T05:44:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac005bb0 con 0x5649973a3080 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aa8030050 con 0x5649973a3080 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ac0838740 10 client.15600 fetch_fsmap learned FSMap version 28 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56499739a260 con 0x5649973a3080 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.290+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac005bb0 con 0x5649973a3080 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3aa801f430 con 0x5649973a3080 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 10 client.15600 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 20 client.15600 populate_metadata read hostname 'smithi038' 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x56499742adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 4 client.15600 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x56499728bec0 con 0x5649973bd690 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x56499742adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3ab77fe640 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x56499742adc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476d902c36738095 server_cookie=a55121b5cdb655ba in_seq=0 out_seq=0 2024-09-19T05:44:49.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.291+0000 7f3a977fe640 10 client.15600 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:44:49.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.306+0000 7f3a977fe640 1 -- 172.21.15.38:0/1343228815 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56499728bec0 con 0x5649973bd690 2024-09-19T05:44:49.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.306+0000 7f3a977fe640 10 client.15600 handle_command_reply: tid=0 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.306+0000 7f3ac0838740 1 client.15600 shutdown 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.306+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 msgr2=0x5649973b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.306+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8004700 tx=0x7f3aa8012da0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 shutdown_connections 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649973bd690 0x56499742adc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649973ad940 0x5649974272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3a977fe640 0 client.15600 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5649973a3080 0x5649973b1f80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 --2- 172.21.15.38:0/1343228815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649973a2760 0x5649973b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 >> 172.21.15.38:0/1343228815 conn(0x56499739ecf0 msgr2=0x5649973aaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 shutdown_connections 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 1 -- 172.21.15.38:0/1343228815 wait complete. 2024-09-19T05:44:49.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3ac0838740 20 client.15600 trim_cache size 0 max 16384 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 14, 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "506af1a6": { 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 14, 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:44:49.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 136.6, 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:44:49.311 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:44:49.312 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:44:49.312 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:44:49.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 msgr2=0x7f3aa00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 0x7f3aa00a5700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00b95f0 tx=0x7f3ab00b7bc0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 msgr2=0x7f3ab8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.307+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0031a20 tx=0x7f3ab0031a50 comp rx=0 tx=0).stop 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 shutdown_connections 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f3aa00a3240 0x7f3aa00a5700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab8106f20 0x7f3ab8074f70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab81027a0 0x7f3ab8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 --2- 172.21.15.38:0/3136446351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab8101df0 0x7f3ab80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 >> 172.21.15.38:0/3136446351 conn(0x7f3ab80fd8b0 msgr2=0x7f3ab80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 shutdown_connections 2024-09-19T05:44:49.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:44:49.308+0000 7f3abe45f640 1 -- 172.21.15.38:0/3136446351 wait complete. 2024-09-19T05:44:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:49 smithi038 ceph-mon[31227]: pgmap v460: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:49.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:49 smithi179 ceph-mon[35781]: pgmap v460: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:49.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:49 smithi047 ceph-mon[35327]: pgmap v460: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:44:50.220 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '506af1a6', quiesce took 3.5 seconds. Will release after: 13.100000000000001 2024-09-19T05:44:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:51 smithi038 ceph-mon[31227]: pgmap v461: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:51 smithi179 ceph-mon[35781]: pgmap v461: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:51.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:51 smithi047 ceph-mon[35327]: pgmap v461: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:44:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:53 smithi038 ceph-mon[31227]: pgmap v462: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:44:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:53 smithi179 ceph-mon[35781]: pgmap v462: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:44:53.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:53 smithi047 ceph-mon[35327]: pgmap v462: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:44:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:55 smithi038 ceph-mon[31227]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:44:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:55 smithi179 ceph-mon[35781]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:44:55.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:55 smithi047 ceph-mon[35327]: pgmap v463: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:44:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:57 smithi038 ceph-mon[31227]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:44:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:57 smithi179 ceph-mon[35781]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:44:57.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:57 smithi047 ceph-mon[35327]: pgmap v464: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:44:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:44:59 smithi179 ceph-mon[35781]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:44:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:44:59 smithi047 ceph-mon[35327]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:44:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:44:59 smithi038 ceph-mon[31227]: pgmap v465: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:45:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:01 smithi179 ceph-mon[35781]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:45:01.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:01 smithi047 ceph-mon[35327]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:45:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:01 smithi038 ceph-mon[31227]: pgmap v466: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:45:03.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:45:03.600 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:45:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:03 smithi038 ceph-mon[31227]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:45:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:03 smithi179 ceph-mon[35781]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:45:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:03 smithi047 ceph-mon[35327]: pgmap v467: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-19T05:45:04.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2497697450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 msgr2=0x7fe19810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2497697450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe19810da40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe190009ca0 tx=0x7fe19002a950 comp rx=0 tx=0).stop 2024-09-19T05:45:04.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2497697450 shutdown_connections 2024-09-19T05:45:04.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2497697450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe19810da40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2497697450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe1981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2497697450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2497697450 >> 172.21.15.38:0/2497697450 conn(0x7fe1980fc7d0 msgr2=0x7fe1980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2497697450 shutdown_connections 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2497697450 wait complete. 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.097+0000 7fe19f1a9640 1 Processor -- start 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 -- start start 2024-09-19T05:45:04.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe198199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe198110fa0 con 0x7fe198102ef0 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe198110e20 con 0x7fe198100e20 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19f1a9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe198111120 con 0x7fe198102570 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19d746640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19d746640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56250/0 (socket says 172.21.15.38:56250) 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19d746640 1 -- 172.21.15.38:0/2022397562 learned_addr learned my addr 172.21.15.38:0/2022397562 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe197fff640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe198199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.098+0000 7fe19cf45640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 -- 172.21.15.38:0/2022397562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 msgr2=0x7fe198199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe198199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 -- 172.21.15.38:0/2022397562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 msgr2=0x7fe198199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 -- 172.21.15.38:0/2022397562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1981a4910 con 0x7fe198102ef0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19cf45640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe197fff640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe198199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19d746640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe190004af0 tx=0x7fe19002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190034040 con 0x7fe198102ef0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe190030920 con 0x7fe198102ef0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1981a4ae0 con 0x7fe198102ef0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190004820 con 0x7fe198102ef0 2024-09-19T05:45:04.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.099+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a4f90 con 0x7fe198102ef0 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.101+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198101200 con 0x7fe198102ef0 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.102+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe190037020 con 0x7fe198102ef0 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.102+0000 7fe195ffb640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 0x7fe1800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.102+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe1900f0be0 con 0x7fe198102ef0 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.102+0000 7fe19cf45640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 0x7fe1800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.103+0000 7fe19cf45640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 0x7fe1800a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe188009c20 tx=0x7fe188005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:04.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.107+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe1900bd540 con 0x7fe198102ef0 2024-09-19T05:45:04.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.319+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1981a5160 con 0x7fe198102ef0 2024-09-19T05:45:04.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.320+0000 7fe195ffb640 1 -- 172.21.15.38:0/2022397562 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fe1900bce40 con 0x7fe198102ef0 2024-09-19T05:45:04.323 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:45:04.324 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:45:04.324 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:45:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 msgr2=0x7fe1800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 0x7fe1800a5780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe188009c20 tx=0x7fe188005f90 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 msgr2=0x7fe1981a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe190004af0 tx=0x7fe19002f5b0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 shutdown_connections 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.322+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe1800a32c0 0x7fe1800a5780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe198102ef0 0x7fe1981a43d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe198102570 0x7fe198199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 --2- 172.21.15.38:0/2022397562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198100e20 0x7fe198199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 >> 172.21.15.38:0/2022397562 conn(0x7fe1980fc7d0 msgr2=0x7fe198106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 shutdown_connections 2024-09-19T05:45:04.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:04.323+0000 7fe19f1a9640 1 -- 172.21.15.38:0/2022397562 wait complete. 2024-09-19T05:45:04.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:04 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2022397562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:04.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:04 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2022397562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:04.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:04 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2022397562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:05.186 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 506af1a6 --release --await' 2024-09-19T05:45:05.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 506af1a6 --release --await 2024-09-19T05:45:05.504 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:45:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:05 smithi179 ceph-mon[35781]: pgmap v468: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:05.700 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:05 smithi038 ceph-mon[31227]: pgmap v468: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:05.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:05 smithi047 ceph-mon[35327]: pgmap v468: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:06.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 -- 172.21.15.38:0/2105813219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 msgr2=0x7f54ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 --2- 172.21.15.38:0/2105813219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac1067e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f549c009900 tx=0x7f549c02a410 comp rx=0 tx=0).stop 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 -- 172.21.15.38:0/2105813219 shutdown_connections 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 --2- 172.21.15.38:0/2105813219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac106f10 0x7f54ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 --2- 172.21.15.38:0/2105813219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac1067e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 --2- 172.21.15.38:0/2105813219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac101de0 0x7f54ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 -- 172.21.15.38:0/2105813219 >> 172.21.15.38:0/2105813219 conn(0x7f54ac0fd8c0 msgr2=0x7f54ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.075+0000 7f54b3928640 1 -- 172.21.15.38:0/2105813219 shutdown_connections 2024-09-19T05:45:06.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 -- 172.21.15.38:0/2105813219 wait complete. 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 Processor -- start 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 -- start start 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac101de0 0x7f54ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac106f10 0x7f54ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac111f70 con 0x7f54ac102790 2024-09-19T05:45:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac111df0 con 0x7f54ac101de0 2024-09-19T05:45:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.076+0000 7f54b3928640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1120f0 con 0x7f54ac106f10 2024-09-19T05:45:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56272/0 (socket says 172.21.15.38:56272) 2024-09-19T05:45:06.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 -- 172.21.15.38:0/3201844941 learned_addr learned my addr 172.21.15.38:0/3201844941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac106f10 0x7f54ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b16c4640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac101de0 0x7f54ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 -- 172.21.15.38:0/3201844941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac106f10 msgr2=0x7f54ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac106f10 0x7f54ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 -- 172.21.15.38:0/3201844941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac101de0 msgr2=0x7f54ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac101de0 0x7f54ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 -- 172.21.15.38:0/3201844941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac1a68b0 con 0x7f54ac102790 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f549c005ab0 tx=0x7f549c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54a27fc640 1 -- 172.21.15.38:0/3201844941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c03f030 con 0x7f54ac102790 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54a27fc640 1 -- 172.21.15.38:0/3201844941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549c003ce0 con 0x7f54ac102790 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac1a6b40 con 0x7f54ac102790 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54a27fc640 1 -- 172.21.15.38:0/3201844941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c035070 con 0x7f54ac102790 2024-09-19T05:45:06.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.077+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1a7020 con 0x7f54ac102790 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54a27fc640 1 -- 172.21.15.38:0/3201844941 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f549c042030 con 0x7f54ac102790 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 Processor -- start 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 -- start start 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb013440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb020630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5653baf78620 con 0x5653bb013980 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653baf1fe50 con 0x5653bb013060 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b5d01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5653baf745e0 con 0x5653bb01e240 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56276/0 (socket says 172.21.15.38:56276) 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.080+0000 7f54b16c4640 1 -- 172.21.15.38:0/437807350 learned_addr learned my addr 172.21.15.38:0/437807350 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:06.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb020630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb013440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 -- 172.21.15.38:0/437807350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 msgr2=0x5653bb020630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb020630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 -- 172.21.15.38:0/437807350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 msgr2=0x5653bb013440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb013440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 -- 172.21.15.38:0/437807350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653baf10d40 con 0x5653bb013980 2024-09-19T05:45:06.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54a27fc640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 0x7f54980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b16c4640 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ac60 tx=0x7f54a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fb13866192e588 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 0x7f54980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54a27fc640 1 -- 172.21.15.38:0/3201844941 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f549c0f0530 con 0x7f54ac102790 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54977fe640 1 -- 172.21.15.38:0/437807350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a801d070 con 0x5653bb013980 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54977fe640 1 -- 172.21.15.38:0/437807350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8018b20 con 0x5653bb013980 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.081+0000 7f54977fe640 1 -- 172.21.15.38:0/437807350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8010be0 con 0x5653bb013980 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- 172.21.15.38:0/437807350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 msgr2=0x5653bb01dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 0x7f54980a5700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007e50 tx=0x7f54a4007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ac60 tx=0x7f54a800f790 comp rx=0 tx=0).stop 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- 172.21.15.38:0/437807350 shutdown_connections 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb020630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013980 0x5653bb01dd00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 --2- 172.21.15.38:0/437807350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb013440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- 172.21.15.38:0/437807350 >> 172.21.15.38:0/437807350 conn(0x5653bb00f5f0 msgr2=0x5653bb00f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- 172.21.15.38:0/437807350 shutdown_connections 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- 172.21.15.38:0/437807350 wait complete. 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.082+0000 7f54b5d01740 1 -- start start 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 Processor -- start 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 0x5653bb01b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb01bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5653bb020e10 con 0x5653bb013060 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653baf00f10 con 0x5653bb013980 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5653bb020f90 con 0x5653bb01e240 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b5d01740 10 client.0 _ll_register_callbacks cb 0x5653baf24db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:45:06.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56286/0 (socket says 172.21.15.38:56286) 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 -- 172.21.15.38:0/1070230 learned_addr learned my addr 172.21.15.38:0/1070230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 0x5653bb01b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 -- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 msgr2=0x5653bb01bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb01bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 -- 172.21.15.38:0/1070230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 msgr2=0x5653bb01b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 0x5653bb01b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b16c4640 1 -- 172.21.15.38:0/1070230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bac87580 con 0x5653bb013060 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.083+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 0x5653bb01b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f54a801b040 tx=0x7f54a8018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3388b3e7d9ca4b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:45:06.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a801d040 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f54a8018810 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8018b20 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54a8031050 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 10 client.15618 fetch_fsmap learned FSMap version 28 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5653bad51960 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8018b20 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f54a8020030 con 0x5653bb013060 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 10 client.15618 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 10 client.15618 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 20 client.15618 populate_metadata read hostname 'smithi038' 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb02dd60 0x5653bb030150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 4 client.15618 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.084+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653baf85b90 con 0x5653bb02dd60 2024-09-19T05:45:06.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.085+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb02dd60 0x5653bb030150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.085+0000 7f5494ff9640 10 client.15618 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.085+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb02dd60 0x5653bb030150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b60a1275c67303f server_cookie=8cf2cac07bf17944 in_seq=0 out_seq=0 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f5494ff9640 1 -- 172.21.15.38:0/1070230 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653baf85b90 con 0x5653bb02dd60 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f5494ff9640 10 client.15618 handle_command_reply: tid=0 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 client.15618 shutdown 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 msgr2=0x5653bb01b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f54a801b040 tx=0x7f54a8018670 comp rx=0 tx=0).stop 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 shutdown_connections 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb02dd60 0x5653bb030150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb01e240 0x5653bb01bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013980 0x5653bb01b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1070230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb01b110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 >> 172.21.15.38:0/1070230 conn(0x5653bb00f5f0 msgr2=0x5653bb015320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f5494ff9640 0 client.15618 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:06.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 shutdown_connections 2024-09-19T05:45:06.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 1 -- 172.21.15.38:0/1070230 wait complete. 2024-09-19T05:45:06.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.086+0000 7f54b5d01740 20 client.15618 trim_cache size 0 max 16384 2024-09-19T05:45:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.103+0000 7f54b5d01740 1 Processor -- start 2024-09-19T05:45:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.103+0000 7f54b5d01740 1 -- start start 2024-09-19T05:45:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb035cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 0x5653bb018370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b0ee0 con 0x5653bb013060 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b2a20 con 0x5653bb01e240 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b5d01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b15b0 con 0x5653bb013980 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56292/0 (socket says 172.21.15.38:56292) 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 -- 172.21.15.38:0/2194200666 learned_addr learned my addr 172.21.15.38:0/2194200666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 0x5653bb018370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb035cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 -- 172.21.15.38:0/2194200666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 msgr2=0x5653bb035cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb035cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 -- 172.21.15.38:0/2194200666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 msgr2=0x5653bb018370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 0x5653bb018370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 -- 172.21.15.38:0/2194200666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653bb00f090 con 0x5653bb013060 2024-09-19T05:45:06.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 0x5653bb018370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f54b16c4640 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f54a8004760 tx=0x7f54a8010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec4fdaab2c3496d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.104+0000 7f5496ffd640 1 -- 172.21.15.38:0/2194200666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a803a070 con 0x5653bb013060 2024-09-19T05:45:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f5496ffd640 1 -- 172.21.15.38:0/2194200666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8028070 con 0x5653bb013060 2024-09-19T05:45:06.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f5496ffd640 1 -- 172.21.15.38:0/2194200666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a801d070 con 0x5653bb013060 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 -- 172.21.15.38:0/2194200666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 msgr2=0x5653bb035780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f54a8004760 tx=0x7f54a8010890 comp rx=0 tx=0).stop 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 -- 172.21.15.38:0/2194200666 shutdown_connections 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb01e240 0x5653bb018370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb035cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 --2- 172.21.15.38:0/2194200666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb013060 0x5653bb035780 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 -- 172.21.15.38:0/2194200666 >> 172.21.15.38:0/2194200666 conn(0x5653bb00f5f0 msgr2=0x5653bb0199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 -- 172.21.15.38:0/2194200666 shutdown_connections 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.105+0000 7f54b5d01740 1 -- 172.21.15.38:0/2194200666 wait complete. 2024-09-19T05:45:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 -- start start 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 Processor -- start 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb0267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb021b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b2350 con 0x5653bb01e240 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b1c80 con 0x5653bb013060 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1aeec0 con 0x5653bb013980 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b5d01740 10 client.0 _ll_register_callbacks cb 0x7f54a8010cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b1ec5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b1ec5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56294/0 (socket says 172.21.15.38:56294) 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.106+0000 7f54b1ec5640 1 -- 172.21.15.38:0/1893711093 learned_addr learned my addr 172.21.15.38:0/1893711093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b0ec3640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb021b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb0267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 -- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 msgr2=0x5653bb021b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb021b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 -- 172.21.15.38:0/1893711093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 msgr2=0x5653bb0267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb0267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 -- 172.21.15.38:0/1893711093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653bb0226f0 con 0x5653bb01e240 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b1ec5640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007390 tx=0x7f54a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e103fea2168f0c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f5494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a400fbc0 con 0x5653bb01e240 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f54a400fd60 con 0x5653bb01e240 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8028070 con 0x5653bb01e240 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54a4028470 con 0x5653bb01e240 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b5d01740 10 client.15624 fetch_fsmap learned FSMap version 28 2024-09-19T05:45:06.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.107+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5653bb098660 con 0x5653bb01e240 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a8028070 con 0x5653bb01e240 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f54a401f660 con 0x5653bb01e240 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 10 client.15624 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 20 client.15624 populate_metadata read hostname 'smithi038' 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb098de0 0x5653bb09b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 4 client.15624 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "506af1a6", "release": true, "await": true}] 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "506af1a6", "release": true, "await": true}) -- 0x5653bb016d30 con 0x5653bb098de0 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.108+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb098de0 0x5653bb09b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.109+0000 7f5494ff9640 10 client.15624 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.109+0000 7f54b16c4640 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb098de0 0x5653bb09b200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3090b2e91561a777 server_cookie=36a5e9a9328802da in_seq=0 out_seq=0 2024-09-19T05:45:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f5494ff9640 1 -- 172.21.15.38:0/1893711093 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5653bb016d30 con 0x5653bb098de0 2024-09-19T05:45:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f5494ff9640 10 client.15624 handle_command_reply: tid=0 2024-09-19T05:45:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 client.15624 shutdown 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 msgr2=0x5653bb022050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007390 tx=0x7f54a4004ab0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 shutdown_connections 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5653bb098de0 0x5653bb09b200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653bb01e240 0x5653bb022050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653bb013980 0x5653bb021b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 --2- 172.21.15.38:0/1893711093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653bb013060 0x5653bb0267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 >> 172.21.15.38:0/1893711093 conn(0x5653bb00f5f0 msgr2=0x5653bb00df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f5494ff9640 0 client.15624 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 shutdown_connections 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 1 -- 172.21.15.38:0/1893711093 wait complete. 2024-09-19T05:45:06.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.112+0000 7f54b5d01740 20 client.15624 trim_cache size 0 max 16384 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 16, 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "506af1a6": { 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 16, 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:45:06.117 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 136.6, 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:06.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:06.119 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:45:06.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 msgr2=0x7f54980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 0x7f54980a5700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007e50 tx=0x7f54a4007520 comp rx=0 tx=0).stop 2024-09-19T05:45:06.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 msgr2=0x7f54ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f549c005ab0 tx=0x7f549c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 shutdown_connections 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f54980a3240 0x7f54980a5700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54ac106f10 0x7f54ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54ac102790 0x7f54ac074a30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 --2- 172.21.15.38:0/3201844941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac101de0 0x7f54ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 >> 172.21.15.38:0/3201844941 conn(0x7f54ac0fd8c0 msgr2=0x7f54ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 shutdown_connections 2024-09-19T05:45:06.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:06.113+0000 7f54b3928640 1 -- 172.21.15.38:0/3201844941 wait complete. 2024-09-19T05:45:06.989 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '506af1a6', total seconds elapsed: 20.3 2024-09-19T05:45:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:07 smithi179 ceph-mon[35781]: pgmap v469: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:45:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:45:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:45:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:45:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:45:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:07 smithi047 ceph-mon[35327]: pgmap v469: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:45:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:45:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:45:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:45:07.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:45:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:07 smithi038 ceph-mon[31227]: pgmap v469: 129 pgs: 129 active+clean; 1.6 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:45:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:45:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:45:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:45:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:45:09.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:09 smithi047 ceph-mon[35327]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-19T05:45:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:09 smithi038 ceph-mon[31227]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-19T05:45:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:09 smithi179 ceph-mon[35781]: pgmap v470: 129 pgs: 129 active+clean; 1.7 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-19T05:45:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:11 smithi047 ceph-mon[35327]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T05:45:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:11 smithi038 ceph-mon[31227]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T05:45:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:11 smithi179 ceph-mon[35781]: pgmap v471: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T05:45:13.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:13 smithi047 ceph-mon[35327]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:45:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:13 smithi038 ceph-mon[31227]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:45:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:13 smithi179 ceph-mon[35781]: pgmap v472: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:45:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:15 smithi047 ceph-mon[35327]: pgmap v473: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-19T05:45:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:15 smithi038 ceph-mon[31227]: pgmap v473: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-19T05:45:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:15 smithi179 ceph-mon[35781]: pgmap v473: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-19T05:45:17.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:17 smithi047 ceph-mon[35327]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-19T05:45:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:17 smithi038 ceph-mon[31227]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-19T05:45:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:17 smithi179 ceph-mon[35781]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-09-19T05:45:19.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:19 smithi047 ceph-mon[35327]: pgmap v475: 129 pgs: 129 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:19.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:19 smithi038 ceph-mon[31227]: pgmap v475: 129 pgs: 129 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:19 smithi179 ceph-mon[35781]: pgmap v475: 129 pgs: 129 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:21 smithi047 ceph-mon[35327]: pgmap v476: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:21 smithi038 ceph-mon[31227]: pgmap v476: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:21 smithi179 ceph-mon[35781]: pgmap v476: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:23.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:23 smithi038 ceph-mon[31227]: pgmap v477: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:23 smithi179 ceph-mon[35781]: pgmap v477: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:23.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:23 smithi047 ceph-mon[35327]: pgmap v477: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:25.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:25 smithi179 ceph-mon[35781]: pgmap v478: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:25.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:25 smithi038 ceph-mon[31227]: pgmap v478: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:25 smithi047 ceph-mon[35327]: pgmap v478: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:27 smithi038 ceph-mon[31227]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:27 smithi179 ceph-mon[35781]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:27.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:27 smithi047 ceph-mon[35327]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:29 smithi038 ceph-mon[31227]: pgmap v480: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:29 smithi179 ceph-mon[35781]: pgmap v480: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:29.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:29 smithi047 ceph-mon[35327]: pgmap v480: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:31.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:31 smithi038 ceph-mon[31227]: pgmap v481: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:31 smithi179 ceph-mon[35781]: pgmap v481: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:31.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:31 smithi047 ceph-mon[35327]: pgmap v481: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:33 smithi038 ceph-mon[31227]: pgmap v482: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:33 smithi179 ceph-mon[35781]: pgmap v482: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:33.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:33 smithi047 ceph-mon[35327]: pgmap v482: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:35 smithi038 ceph-mon[31227]: pgmap v483: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:35 smithi179 ceph-mon[35781]: pgmap v483: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:35.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:35 smithi047 ceph-mon[35327]: pgmap v483: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:37.819 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:45:37.819 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T05:45:37.820 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:45:37.851 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:37 smithi038 ceph-mon[31227]: pgmap v484: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:37 smithi179 ceph-mon[35781]: pgmap v484: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:37 smithi047 ceph-mon[35327]: pgmap v484: 129 pgs: 129 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:45:38.094 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:45:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 -- 172.21.15.38:0/2412056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 msgr2=0x7f57181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 --2- 172.21.15.38:0/2412056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f57181067e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f570c009900 tx=0x7f570c02a410 comp rx=0 tx=0).stop 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 -- 172.21.15.38:0/2412056091 shutdown_connections 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 --2- 172.21.15.38:0/2412056091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718106f10 0x7f571810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 --2- 172.21.15.38:0/2412056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f57181067e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 --2- 172.21.15.38:0/2412056091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718101de0 0x7f57181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 -- 172.21.15.38:0/2412056091 >> 172.21.15.38:0/2412056091 conn(0x7f57180fd8c0 msgr2=0x7f57180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 -- 172.21.15.38:0/2412056091 shutdown_connections 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 -- 172.21.15.38:0/2412056091 wait complete. 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.615+0000 7f571cea5640 1 Processor -- start 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 -- start start 2024-09-19T05:45:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718101de0 0x7f5718198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 0x7f571819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5718111f90 con 0x7f5718102790 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5718111e10 con 0x7f5718101de0 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571cea5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5718112110 con 0x7f5718106f10 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56296/0 (socket says 172.21.15.38:56296) 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 -- 172.21.15.38:0/2348825886 learned_addr learned my addr 172.21.15.38:0/2348825886 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f571659c640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718101de0 0x7f5718198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5716d9d640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 0x7f571819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:38.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 -- 172.21.15.38:0/2348825886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 msgr2=0x7f571819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 0x7f571819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 -- 172.21.15.38:0/2348825886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718101de0 msgr2=0x7f5718198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718101de0 0x7f5718198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5715d9b640 1 -- 172.21.15.38:0/2348825886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571819d4c0 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.616+0000 7f5716d9d640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 0x7f571819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f5715d9b640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f570c005ab0 tx=0x7f570c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c03f030 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571819d750 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570c003ce0 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c035070 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.617+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a5090 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.618+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57181032c0 con 0x7f5718102790 2024-09-19T05:45:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.620+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f570c042030 con 0x7f5718102790 2024-09-19T05:45:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.620+0000 7f57077fe640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 0x7f57000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.621+0000 7f571659c640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 0x7f57000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.621+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f570c0f0900 con 0x7f5718102790 2024-09-19T05:45:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.621+0000 7f571659c640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 0x7f57000a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5708009990 tx=0x7f5708009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.624+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f570c0bd200 con 0x7f5718102790 2024-09-19T05:45:38.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.835+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57181021c0 con 0x7f5718102790 2024-09-19T05:45:38.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.835+0000 7f57077fe640 1 -- 172.21.15.38:0/2348825886 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f57181021c0 con 0x7f5718102790 2024-09-19T05:45:38.837 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:45:38.837 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:45:38.838 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:45:38.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 msgr2=0x7f57000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:38.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 0x7f57000a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5708009990 tx=0x7f5708009110 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 msgr2=0x7f5718198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f570c005ab0 tx=0x7f570c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 shutdown_connections 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f57000a3240 0x7f57000a5700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5718106f10 0x7f571819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5718102790 0x7f5718198a50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 --2- 172.21.15.38:0/2348825886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5718101de0 0x7f5718198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 >> 172.21.15.38:0/2348825886 conn(0x7f57180fd8c0 msgr2=0x7f57180fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 shutdown_connections 2024-09-19T05:45:38.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:38.839+0000 7f571cea5640 1 -- 172.21.15.38:0/2348825886 wait complete. 2024-09-19T05:45:39.627 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T05:45:39.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T05:45:39.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:39 smithi038 ceph-mon[31227]: pgmap v485: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:39.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:39 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2348825886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:39.928 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:45:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:39 smithi179 ceph-mon[35781]: pgmap v485: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:39 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2348825886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:39 smithi047 ceph-mon[35327]: pgmap v485: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:45:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:39 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2348825886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:45:40.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 -- 172.21.15.38:0/330649822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f734410e3c0 msgr2=0x7f7344110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 --2- 172.21.15.38:0/330649822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f734410e3c0 0x7f7344110810 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f733c009ca0 tx=0x7f733c02a950 comp rx=0 tx=0).stop 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 -- 172.21.15.38:0/330649822 shutdown_connections 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 --2- 172.21.15.38:0/330649822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f734410e3c0 0x7f7344110810 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 --2- 172.21.15.38:0/330649822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7344074f40 0x7f734410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 --2- 172.21.15.38:0/330649822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 0x7f7344074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 -- 172.21.15.38:0/330649822 >> 172.21.15.38:0/330649822 conn(0x7f73440fd8c0 msgr2=0x7f73440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 -- 172.21.15.38:0/330649822 shutdown_connections 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.478+0000 7f734c34e640 1 -- 172.21.15.38:0/330649822 wait complete. 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 Processor -- start 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 -- start start 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 0x7f734419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734410e3c0 0x7f734419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114170 con 0x7f7344074f40 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7344113ff0 con 0x7f7344074620 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734c34e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73441142f0 con 0x7f734410e3c0 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56318/0 (socket says 172.21.15.38:56318) 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 -- 172.21.15.38:0/583490439 learned_addr learned my addr 172.21.15.38:0/583490439 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734a0ea640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 0x7f734419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f734a8eb640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734410e3c0 0x7f734419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 -- 172.21.15.38:0/583490439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734410e3c0 msgr2=0x7f734419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734410e3c0 0x7f734419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 -- 172.21.15.38:0/583490439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 msgr2=0x7f734419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 0x7f734419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.479+0000 7f73498e9640 1 -- 172.21.15.38:0/583490439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419f580 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f73498e9640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f733400c970 tx=0x7f733400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f733b7fe640 1 -- 172.21.15.38:0/583490439 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334007bb0 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f733b7fe640 1 -- 172.21.15.38:0/583490439 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7334007d50 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419f870 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f733b7fe640 1 -- 172.21.15.38:0/583490439 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400f450 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.480+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441a7150 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 Processor -- start 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f733b7fe640 1 -- 172.21.15.38:0/583490439 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7334004f40 con 0x7f7344074f40 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 -- start start 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a768b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a773940 0x55af8a775d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55af8a6ce2a0 con 0x55af8a769080 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55af8a675b10 con 0x55af8a768760 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.483+0000 7f734e727740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af8a6ca260 con 0x55af8a773940 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56332/0 (socket says 172.21.15.38:56332) 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a8eb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a768b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 -- 172.21.15.38:0/1679823989 learned_addr learned my addr 172.21.15.38:0/1679823989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f73498e9640 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a773940 0x55af8a775d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 -- 172.21.15.38:0/1679823989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a773940 msgr2=0x55af8a775d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a773940 0x55af8a775d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 -- 172.21.15.38:0/1679823989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 msgr2=0x55af8a768b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a768b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 -- 172.21.15.38:0/1679823989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8a43fb90 con 0x55af8a769080 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f733b7fe640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 0x7f73300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a8eb640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 0x7f73300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f733b7fe640 1 -- 172.21.15.38:0/583490439 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f73340cf930 con 0x7f7344074f40 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f734a0ea640 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f734000ad00 tx=0x7f734000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c0197838926868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f7338dea640 1 -- 172.21.15.38:0/1679823989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734001d070 con 0x55af8a769080 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f7338dea640 1 -- 172.21.15.38:0/1679823989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7340018b20 con 0x55af8a769080 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.484+0000 7f7338dea640 1 -- 172.21.15.38:0/1679823989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340010bc0 con 0x55af8a769080 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 -- 172.21.15.38:0/1679823989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 msgr2=0x55af8a773400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734a8eb640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 0x7f73300a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fe0 tx=0x7f733c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f734000ad00 tx=0x7f734000f790 comp rx=0 tx=0).stop 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 -- 172.21.15.38:0/1679823989 shutdown_connections 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a773940 0x55af8a775d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a769080 0x55af8a773400 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 --2- 172.21.15.38:0/1679823989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a768b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 -- 172.21.15.38:0/1679823989 >> 172.21.15.38:0/1679823989 conn(0x55af8a764cf0 msgr2=0x55af8a7650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 -- 172.21.15.38:0/1679823989 shutdown_connections 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.485+0000 7f734e727740 1 -- 172.21.15.38:0/1679823989 wait complete. 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 -- start start 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 Processor -- start 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a768760 0x55af8a7707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a769080 0x55af8a770cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55af8a776510 con 0x55af8a773940 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55af8a6566e0 con 0x55af8a769080 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af8a776690 con 0x55af8a768760 2024-09-19T05:45:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734e727740 10 client.0 _ll_register_callbacks cb 0x55af8a67aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56340/0 (socket says 172.21.15.38:56340) 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 -- 172.21.15.38:0/2421572166 learned_addr learned my addr 172.21.15.38:0/2421572166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a0ea640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a768760 0x55af8a7707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f73498e9640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a769080 0x55af8a770cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 -- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a768760 msgr2=0x55af8a7707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a768760 0x55af8a7707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 -- 172.21.15.38:0/2421572166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a769080 msgr2=0x55af8a770cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a769080 0x55af8a770cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.486+0000 7f734a8eb640 1 -- 172.21.15.38:0/2421572166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8a77f980 con 0x55af8a773940 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f734a8eb640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f733c02f830 tx=0x7f733c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2935f17284285f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:45:40.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c055060 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f733c0043d0 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7340018b20 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f733c05c050 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f734e727740 10 client.15642 fetch_fsmap learned FSMap version 28 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af8a782310 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7340018b20 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.487+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f733c03a430 con 0x55af8a773940 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 10 client.15642 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 20 client.15642 populate_metadata read hostname 'smithi038' 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7837d0 0x55af8a785c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 4 client.15642 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af8a6db9d0 con 0x55af8a7837d0 2024-09-19T05:45:40.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734a0ea640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7837d0 0x55af8a785c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f734a0ea640 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7837d0 0x55af8a785c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730a4d47068769c5 server_cookie=6dcdf9c3245ab43e in_seq=0 out_seq=0 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.488+0000 7f732dffb640 10 client.15642 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f732dffb640 1 -- 172.21.15.38:0/2421572166 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af8a6db9d0 con 0x55af8a7837d0 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f732dffb640 10 client.15642 handle_command_reply: tid=0 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 client.15642 shutdown 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 msgr2=0x55af8a771230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f733c02f830 tx=0x7f733c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 shutdown_connections 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7837d0 0x55af8a785c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a771230 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a769080 0x55af8a770cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f732dffb640 0 client.15642 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:40.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 --2- 172.21.15.38:0/2421572166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a768760 0x55af8a7707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 >> 172.21.15.38:0/2421572166 conn(0x55af8a764cf0 msgr2=0x55af8a76aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 shutdown_connections 2024-09-19T05:45:40.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 1 -- 172.21.15.38:0/2421572166 wait complete. 2024-09-19T05:45:40.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.489+0000 7f734e727740 20 client.15642 trim_cache size 0 max 16384 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.506+0000 7f734e727740 1 Processor -- start 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.506+0000 7f734e727740 1 -- start start 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a76b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a772680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73441b0ed0 con 0x55af8a773940 2024-09-19T05:45:40.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f73441b2a10 con 0x55af8a768760 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734e727740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73441b15a0 con 0x55af8a769080 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f73498e9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a772680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56350/0 (socket says 172.21.15.38:56350) 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 -- 172.21.15.38:0/853186545 learned_addr learned my addr 172.21.15.38:0/853186545 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a0ea640 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a76b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 -- 172.21.15.38:0/853186545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 msgr2=0x55af8a772680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a772680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 -- 172.21.15.38:0/853186545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 msgr2=0x55af8a76b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a76b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 -- 172.21.15.38:0/853186545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8a764b20 con 0x55af8a773940 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f734a8eb640 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f733c006010 tx=0x7f733c0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f39ad73ea70ddfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f732ffff640 1 -- 172.21.15.38:0/853186545 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c04a020 con 0x55af8a773940 2024-09-19T05:45:40.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f732ffff640 1 -- 172.21.15.38:0/853186545 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c03a430 con 0x55af8a773940 2024-09-19T05:45:40.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.507+0000 7f732ffff640 1 -- 172.21.15.38:0/853186545 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c055070 con 0x55af8a773940 2024-09-19T05:45:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- 172.21.15.38:0/853186545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 msgr2=0x55af8a76f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f733c006010 tx=0x7f733c0099c0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- 172.21.15.38:0/853186545 shutdown_connections 2024-09-19T05:45:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a773940 0x55af8a76f080 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a772680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 --2- 172.21.15.38:0/853186545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a768760 0x55af8a76b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- 172.21.15.38:0/853186545 >> 172.21.15.38:0/853186545 conn(0x55af8a764cf0 msgr2=0x55af8a763e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- 172.21.15.38:0/853186545 shutdown_connections 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- 172.21.15.38:0/853186545 wait complete. 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.508+0000 7f734e727740 1 -- start start 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 Processor -- start 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a778710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a773940 0x55af8a7e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f73441b2340 con 0x55af8a768760 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f73441b1c70 con 0x55af8a773940 2024-09-19T05:45:40.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73441aeeb0 con 0x55af8a769080 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734e727740 10 client.0 _ll_register_callbacks cb 0x7f733c02f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56352/0 (socket says 172.21.15.38:56352) 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 -- 172.21.15.38:0/3636893027 learned_addr learned my addr 172.21.15.38:0/3636893027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a8eb640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a773940 0x55af8a7e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f73498e9640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a778710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 -- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 msgr2=0x55af8a778710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a778710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 -- 172.21.15.38:0/3636893027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a773940 msgr2=0x55af8a7e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a773940 0x55af8a7e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.509+0000 7f734a0ea640 1 -- 172.21.15.38:0/3636893027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8a765980 con 0x55af8a768760 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734a0ea640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f734000acd0 tx=0x7f7340010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bccb4ae49a3f14c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:40.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734001d040 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7340010ca0 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733c03a430 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f734003b050 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af8a760260 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c03a430 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f73400053d0 con 0x55af8a768760 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 10 client.15648 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 20 client.15648 populate_metadata read hostname 'smithi038' 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7ef310 0x55af8a7f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 4 client.15648 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T05:45:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.510+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x55af8a6d5d20 con 0x55af8a7ef310 2024-09-19T05:45:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.511+0000 7f73498e9640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7ef310 0x55af8a7f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.511+0000 7f73498e9640 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7ef310 0x55af8a7f1730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34123a0460f5cb83 server_cookie=7d32294cdba461a in_seq=0 out_seq=0 2024-09-19T05:45:40.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.511+0000 7f732dffb640 10 client.15648 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f732dffb640 1 -- 172.21.15.38:0/3636893027 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55af8a6d5d20 con 0x55af8a7ef310 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f732dffb640 10 client.15648 handle_command_reply: tid=0 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 client.15648 shutdown 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 msgr2=0x55af8a7781d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f734000acd0 tx=0x7f7340010840 comp rx=0 tx=0).stop 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 shutdown_connections 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55af8a7ef310 0x55af8a7f1730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af8a773940 0x55af8a7e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af8a769080 0x55af8a778710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 --2- 172.21.15.38:0/3636893027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55af8a768760 0x55af8a7781d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 >> 172.21.15.38:0/3636893027 conn(0x55af8a764cf0 msgr2=0x55af8a77e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f732dffb640 0 client.15648 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 shutdown_connections 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 1 -- 172.21.15.38:0/3636893027 wait complete. 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.523+0000 7f734e727740 20 client.15648 trim_cache size 0 max 16384 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:45:40.526 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 18, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "dfdb7625": { 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 18, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.8, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:45:40.527 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:45:40.528 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 msgr2=0x7f73300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 0x7f73300a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fe0 tx=0x7f733c005c50 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 msgr2=0x7f734419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f733400c970 tx=0x7f733400cd70 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 shutdown_connections 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f73300a3240 0x7f73300a5700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734410e3c0 0x7f734419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7344074f40 0x7f734419ab10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 --2- 172.21.15.38:0/583490439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7344074620 0x7f734419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:40.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 >> 172.21.15.38:0/583490439 conn(0x7f73440fd8c0 msgr2=0x7f73440fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:40.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 shutdown_connections 2024-09-19T05:45:40.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:45:40.524+0000 7f734c34e640 1 -- 172.21.15.38:0/583490439 wait complete. 2024-09-19T05:45:41.447 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dfdb7625', quiesce took 3.6 seconds. Will release after: 26.2 2024-09-19T05:45:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:41 smithi038 ceph-mon[31227]: pgmap v486: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:41 smithi179 ceph-mon[35781]: pgmap v486: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:41 smithi047 ceph-mon[35327]: pgmap v486: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:45:43.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:43 smithi038 ceph-mon[31227]: pgmap v487: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:43 smithi179 ceph-mon[35781]: pgmap v487: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:43.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:43 smithi047 ceph-mon[35327]: pgmap v487: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:45:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:45 smithi038 ceph-mon[31227]: pgmap v488: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:45 smithi179 ceph-mon[35781]: pgmap v488: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:45 smithi047 ceph-mon[35327]: pgmap v488: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:45:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:47 smithi038 ceph-mon[31227]: pgmap v489: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:45:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:47 smithi179 ceph-mon[35781]: pgmap v489: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:45:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:47 smithi047 ceph-mon[35327]: pgmap v489: 129 pgs: 129 active+clean; 2.0 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:45:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:49 smithi038 ceph-mon[31227]: pgmap v490: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:49 smithi179 ceph-mon[35781]: pgmap v490: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:49.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:49 smithi047 ceph-mon[35327]: pgmap v490: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:45:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:51 smithi038 ceph-mon[31227]: pgmap v491: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:45:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:51 smithi179 ceph-mon[35781]: pgmap v491: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:45:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:51 smithi047 ceph-mon[35327]: pgmap v491: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:45:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:53 smithi038 ceph-mon[31227]: pgmap v492: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-19T05:45:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:53 smithi179 ceph-mon[35781]: pgmap v492: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-19T05:45:53.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:53 smithi047 ceph-mon[35327]: pgmap v492: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-19T05:45:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:55 smithi038 ceph-mon[31227]: pgmap v493: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:55 smithi179 ceph-mon[35781]: pgmap v493: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:55 smithi047 ceph-mon[35327]: pgmap v493: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:45:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:57 smithi038 ceph-mon[31227]: pgmap v494: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:45:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:57 smithi179 ceph-mon[35781]: pgmap v494: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:45:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:57 smithi047 ceph-mon[35327]: pgmap v494: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:45:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:45:59 smithi038 ceph-mon[31227]: pgmap v495: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:45:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:45:59 smithi179 ceph-mon[35781]: pgmap v495: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:45:59.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:45:59 smithi047 ceph-mon[35327]: pgmap v495: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:46:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:01 smithi038 ceph-mon[31227]: pgmap v496: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:01 smithi179 ceph-mon[35781]: pgmap v496: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:01.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:01 smithi047 ceph-mon[35327]: pgmap v496: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:03.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:03 smithi038 ceph-mon[31227]: pgmap v497: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:03 smithi179 ceph-mon[35781]: pgmap v497: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:03.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:03 smithi047 ceph-mon[35327]: pgmap v497: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:05 smithi038 ceph-mon[31227]: pgmap v498: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:05 smithi179 ceph-mon[35781]: pgmap v498: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:05.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:05 smithi047 ceph-mon[35327]: pgmap v498: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:07.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:46:07.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:07 smithi038 ceph-mon[31227]: pgmap v499: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:46:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:07 smithi179 ceph-mon[35781]: pgmap v499: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:46:07.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:07 smithi047 ceph-mon[35327]: pgmap v499: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:07.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 -- 172.21.15.38:0/3170194011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334106f10 msgr2=0x7f633410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 --2- 172.21.15.38:0/3170194011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334106f10 0x7f633410ea90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f632c009900 tx=0x7f632c02a410 comp rx=0 tx=0).stop 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 -- 172.21.15.38:0/3170194011 shutdown_connections 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 --2- 172.21.15.38:0/3170194011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334106f10 0x7f633410ea90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 --2- 172.21.15.38:0/3170194011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334102790 0x7f63341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 --2- 172.21.15.38:0/3170194011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334101de0 0x7f63341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 -- 172.21.15.38:0/3170194011 >> 172.21.15.38:0/3170194011 conn(0x7f63340fd8c0 msgr2=0x7f63340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 -- 172.21.15.38:0/3170194011 shutdown_connections 2024-09-19T05:46:08.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.403+0000 7f63393e8640 1 -- 172.21.15.38:0/3170194011 wait complete. 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 Processor -- start 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 -- start start 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334101de0 0x7f63341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334106f10 0x7f633419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6334111f70 con 0x7f6334102790 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6334111df0 con 0x7f6334106f10 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63393e8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f63341120f0 con 0x7f6334101de0 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63327fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63327fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58942/0 (socket says 172.21.15.38:58942) 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f63327fc640 1 -- 172.21.15.38:0/3656675654 learned_addr learned my addr 172.21.15.38:0/3656675654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:08.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.404+0000 7f6332ffd640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334101de0 0x7f63341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63337fe640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334106f10 0x7f633419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 -- 172.21.15.38:0/3656675654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334101de0 msgr2=0x7f63341983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334101de0 0x7f63341983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 -- 172.21.15.38:0/3656675654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334106f10 msgr2=0x7f633419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334106f10 0x7f633419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 -- 172.21.15.38:0/3656675654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633419d3a0 con 0x7f6334102790 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63327fc640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f632800d950 tx=0x7f632800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328017030 con 0x7f6334102790 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6328004020 con 0x7f6334102790 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633419d690 con 0x7f6334102790 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280047a0 con 0x7f6334102790 2024-09-19T05:46:08.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.405+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341a4f70 con 0x7f6334102790 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.406+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63341021c0 con 0x7f6334102790 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.407+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f63280041d0 con 0x7f6334102790 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.408+0000 7f631bfff640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 0x7f631c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.408+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f63280cf650 con 0x7f6334102790 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.408+0000 7f6332ffd640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 0x7f631c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.409+0000 7f6332ffd640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 0x7f631c0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fd0 tx=0x7f6324009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:08.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.412+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f632809bfb0 con 0x7f6334102790 2024-09-19T05:46:08.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.628+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6334109bc0 con 0x7f6334102790 2024-09-19T05:46:08.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.629+0000 7f631bfff640 1 -- 172.21.15.38:0/3656675654 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f632809b8b0 con 0x7f6334102790 2024-09-19T05:46:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:46:08.631 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:46:08.631 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 msgr2=0x7f631c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 0x7f631c0a5700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fd0 tx=0x7f6324009680 comp rx=0 tx=0).stop 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 msgr2=0x7f6334198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f632800d950 tx=0x7f632800de10 comp rx=0 tx=0).stop 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 shutdown_connections 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f631c0a3240 0x7f631c0a5700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6334106f10 0x7f633419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6334102790 0x7f6334198930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 --2- 172.21.15.38:0/3656675654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6334101de0 0x7f63341983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:08.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 >> 172.21.15.38:0/3656675654 conn(0x7f63340fd8c0 msgr2=0x7f63340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:08.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 shutdown_connections 2024-09-19T05:46:08.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:08.631+0000 7f63393e8640 1 -- 172.21.15.38:0/3656675654 wait complete. 2024-09-19T05:46:08.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:46:08.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:46:08.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:46:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:46:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:46:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:46:08.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:46:08.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:46:08.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:46:09.415 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id dfdb7625 --release --await' 2024-09-19T05:46:09.415 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id dfdb7625 --release --await 2024-09-19T05:46:09.683 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:09.806 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:09 smithi038 ceph-mon[31227]: pgmap v500: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:09.806 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:09 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3656675654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:09 smithi179 ceph-mon[35781]: pgmap v500: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:09 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3656675654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:09.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:09 smithi047 ceph-mon[35327]: pgmap v500: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:09.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:09 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3656675654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:10.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/663729956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8102790 msgr2=0x7f4df81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/663729956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8102790 0x7f4df81067e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009900 tx=0x7f4dec02a410 comp rx=0 tx=0).stop 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/663729956 shutdown_connections 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/663729956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/663729956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8102790 0x7f4df81067e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/663729956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8101de0 0x7f4df81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/663729956 >> 172.21.15.38:0/663729956 conn(0x7f4df80fd8c0 msgr2=0x7f4df80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/663729956 shutdown_connections 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/663729956 wait complete. 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 Processor -- start 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.223+0000 7f4dfe8e2640 1 -- start start 2024-09-19T05:46:10.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8102790 0x7f4df8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8111f90 con 0x7f4df8101de0 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8111e10 con 0x7f4df8102790 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfe8e2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112110 con 0x7f4df8106f10 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58966/0 (socket says 172.21.15.38:58966) 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 -- 172.21.15.38:0/3597895655 learned_addr learned my addr 172.21.15.38:0/3597895655 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df77fe640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8102790 0x7f4df8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 -- 172.21.15.38:0/3597895655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 msgr2=0x7f4df819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 -- 172.21.15.38:0/3597895655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8102790 msgr2=0x7f4df8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8102790 0x7f4df8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4df7fff640 1 -- 172.21.15.38:0/3597895655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df819d4c0 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.224+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4df7fff640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c890 tx=0x7f4de800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4df57fa640 1 -- 172.21.15.38:0/3597895655 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004220 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4df57fa640 1 -- 172.21.15.38:0/3597895655 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de80043c0 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df819d7b0 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4df57fa640 1 -- 172.21.15.38:0/3597895655 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80049e0 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.225+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a5090 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4df57fa640 1 -- 172.21.15.38:0/3597895655 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4de80079b0 con 0x7f4df8101de0 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 Processor -- start 2024-09-19T05:46:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 -- start start 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa61670110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61670650 0x55fa61672a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa615c6620 con 0x55fa61660f60 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fa6156de50 con 0x55fa61670650 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa615c25e0 con 0x55fa61661880 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58974/0 (socket says 172.21.15.38:58974) 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.228+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2543513671 learned_addr learned my addr 172.21.15.38:0/2543513671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4df7fff640 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa61670110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4df77fe640 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61670650 0x55fa61672a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4df57fa640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 0x7f4de00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2543513671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 msgr2=0x55fa61670110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa61670110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2543513671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61670650 msgr2=0x55fa61672a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61670650 0x55fa61672a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2543513671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa6155ed40 con 0x55fa61660f60 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 0x7f4de00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4df57fa640 1 -- 172.21.15.38:0/3597895655 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4de80cfd30 con 0x7f4df8101de0 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4df000c550 tx=0x7f4df000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0eca4b0bcd8b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dde7fc640 1 -- 172.21.15.38:0/2543513671 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000a600 con 0x55fa61660f60 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dde7fc640 1 -- 172.21.15.38:0/2543513671 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df000ac80 con 0x55fa61660f60 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.229+0000 7f4dde7fc640 1 -- 172.21.15.38:0/2543513671 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df00179e0 con 0x55fa61660f60 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2543513671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 msgr2=0x55fa61661340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 0x7f4de00a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4df001f000 tx=0x7f4df0018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4df000c550 tx=0x7f4df000ca10 comp rx=0 tx=0).stop 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2543513671 shutdown_connections 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61670650 0x55fa61672a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa61670110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2543513671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61661340 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2543513671 >> 172.21.15.38:0/2543513671 conn(0x55fa6165d5f0 msgr2=0x55fa6165d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2543513671 shutdown_connections 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2543513671 wait complete. 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.230+0000 7f4e00cbb740 1 -- start start 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 Processor -- start 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61660f60 0x55fa6166d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa6166db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa61673280 con 0x55fa61670650 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fa6154ef10 con 0x55fa61661880 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa61673400 con 0x55fa61660f60 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4e00cbb740 10 client.0 _ll_register_callbacks cb 0x55fa61572db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4df77fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa6166db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58986/0 (socket says 172.21.15.38:58986) 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2850512929 learned_addr learned my addr 172.21.15.38:0/2850512929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4df7fff640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61660f60 0x55fa6166d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61660f60 msgr2=0x55fa6166d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61660f60 0x55fa6166d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2850512929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 msgr2=0x55fa6166db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa6166db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4dfce7f640 1 -- 172.21.15.38:0/2850512929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa612d5580 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.231+0000 7f4df77fe640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa6166db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4df0017740 tx=0x7f4df00179f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae3a62183d2f07b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0009cf0 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df000ac80 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4df000a610 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4df0038050 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4e00cbb740 10 client.15666 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa6139f960 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df000ac80 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.232+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4df0018920 con 0x55fa61670650 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 10 client.15666 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 20 client.15666 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa61680190 0x55fa61682580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 4 client.15666 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa615d3b90 con 0x55fa61680190 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4df7fff640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa61680190 0x55fa61682580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4df7fff640 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa61680190 0x55fa61682580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558cd3ec4b28a637 server_cookie=dba100a25fe0c83f in_seq=0 out_seq=0 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.233+0000 7f4dd77fe640 10 client.15666 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4dd77fe640 1 -- 172.21.15.38:0/2850512929 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa615d3b90 con 0x55fa61680190 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4dd77fe640 10 client.15666 handle_command_reply: tid=0 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 client.15666 shutdown 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 msgr2=0x55fa6166e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4df0017740 tx=0x7f4df00179f0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 shutdown_connections 2024-09-19T05:46:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa61680190 0x55fa61682580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166e070 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa6166db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4dd77fe640 0 client.15666 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/2850512929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61660f60 0x55fa6166d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 >> 172.21.15.38:0/2850512929 conn(0x55fa6165d5f0 msgr2=0x55fa61663350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 shutdown_connections 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 1 -- 172.21.15.38:0/2850512929 wait complete. 2024-09-19T05:46:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.234+0000 7f4e00cbb740 20 client.15666 trim_cache size 0 max 16384 2024-09-19T05:46:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.251+0000 7f4e00cbb740 1 Processor -- start 2024-09-19T05:46:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.251+0000 7f4e00cbb740 1 -- start start 2024-09-19T05:46:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 0x55fa61687d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa616882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81acba0 con 0x55fa61670650 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ae6e0 con 0x55fa61660f60 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ad270 con 0x55fa61661880 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58988/0 (socket says 172.21.15.38:58988) 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 -- 172.21.15.38:0/3493831843 learned_addr learned my addr 172.21.15.38:0/3493831843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4df77fe640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa616882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4df7fff640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 0x55fa61687d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 -- 172.21.15.38:0/3493831843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 msgr2=0x55fa616882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa616882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 -- 172.21.15.38:0/3493831843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 msgr2=0x55fa61687d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 0x55fa61687d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 -- 172.21.15.38:0/3493831843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa6165e550 con 0x55fa61670650 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4df77fe640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa616882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4df7fff640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 0x55fa61687d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.252+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4df0036040 tx=0x7f4df0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7856f434481b385c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4dddffb640 1 -- 172.21.15.38:0/3493831843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0017a60 con 0x55fa61670650 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4dddffb640 1 -- 172.21.15.38:0/3493831843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0009cf0 con 0x55fa61670650 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4dddffb640 1 -- 172.21.15.38:0/3493831843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df00173a0 con 0x55fa61670650 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 -- 172.21.15.38:0/3493831843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 msgr2=0x55fa6166a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4df0036040 tx=0x7f4df0004b50 comp rx=0 tx=0).stop 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 -- 172.21.15.38:0/3493831843 shutdown_connections 2024-09-19T05:46:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61670650 0x55fa6166a850 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61661880 0x55fa616882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/3493831843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61660f60 0x55fa61687d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 -- 172.21.15.38:0/3493831843 >> 172.21.15.38:0/3493831843 conn(0x55fa6165d5f0 msgr2=0x55fa6166bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 -- 172.21.15.38:0/3493831843 shutdown_connections 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.253+0000 7f4e00cbb740 1 -- 172.21.15.38:0/3493831843 wait complete. 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 -- start start 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 Processor -- start 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa61674290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61670650 0x55fa61674a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ae010 con 0x55fa61660f60 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81ad940 con 0x55fa61661880 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81aab80 con 0x55fa61670650 2024-09-19T05:46:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4e00cbb740 10 client.0 _ll_register_callbacks cb 0x7f4df000c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58998/0 (socket says 172.21.15.38:58998) 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4df7fff640 1 -- 172.21.15.38:0/251981714 learned_addr learned my addr 172.21.15.38:0/251981714 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.254+0000 7f4dfce7f640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61670650 0x55fa61674a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df77fe640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa61674290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 -- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61670650 msgr2=0x55fa61674a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61670650 0x55fa61674a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 -- 172.21.15.38:0/251981714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 msgr2=0x55fa61674290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa61674290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 -- 172.21.15.38:0/251981714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa616e9d30 con 0x55fa61660f60 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4df7fff640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de809bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299dc27fc225fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de801d040 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4de809a290 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df0009cf0 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4de809c690 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4e00cbb740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa616eaf60 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.255+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df0009cf0 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4de8007ca0 con 0x55fa61660f60 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 10 client.15672 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 20 client.15672 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa616eb680 0x55fa616eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 4 client.15672 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dfdb7625", "release": true, "await": true}] 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dfdb7625", "release": true, "await": true}) -- 0x55fa615ce0a0 con 0x55fa616eb680 2024-09-19T05:46:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4df77fe640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa616eb680 0x55fa616eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4df77fe640 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa616eb680 0x55fa616eda70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3f8fc61cf27f75 server_cookie=859a8139aecc1bb5 in_seq=0 out_seq=0 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.256+0000 7f4dd77fe640 10 client.15672 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.260+0000 7f4dd77fe640 1 -- 172.21.15.38:0/251981714 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fa615ce0a0 con 0x55fa616eb680 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.260+0000 7f4dd77fe640 10 client.15672 handle_command_reply: tid=0 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.260+0000 7f4e00cbb740 1 client.15672 shutdown 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 msgr2=0x55fa61678f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4de8000c00 tx=0x7f4de809bb90 comp rx=0 tx=0).stop 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 shutdown_connections 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55fa616eb680 0x55fa616eda70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa61670650 0x55fa61674a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa61661880 0x55fa61674290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4dd77fe640 0 client.15672 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 --2- 172.21.15.38:0/251981714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa61660f60 0x55fa61678f60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 >> 172.21.15.38:0/251981714 conn(0x55fa6165d5f0 msgr2=0x55fa61662a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 shutdown_connections 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 1 -- 172.21.15.38:0/251981714 wait complete. 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.261+0000 7f4e00cbb740 20 client.15672 trim_cache size 0 max 16384 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 20, 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "dfdb7625": { 2024-09-19T05:46:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 20, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.8, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:46:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:10.274 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 msgr2=0x7f4de00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 0x7f4de00a5700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4df001f000 tx=0x7f4df0018000 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 msgr2=0x7f4df8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c890 tx=0x7f4de800cd50 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 shutdown_connections 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4de00a3240 0x7f4de00a5700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4df8106f10 0x7f4df819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4df8102790 0x7f4df8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 --2- 172.21.15.38:0/3597895655 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4df8101de0 0x7f4df8198510 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 >> 172.21.15.38:0/3597895655 conn(0x7f4df80fd8c0 msgr2=0x7f4df80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:10.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 shutdown_connections 2024-09-19T05:46:10.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:10.262+0000 7f4dfe8e2640 1 -- 172.21.15.38:0/3597895655 wait complete. 2024-09-19T05:46:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:10 smithi179 ceph-mon[35781]: pgmap v501: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:10.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:10 smithi047 ceph-mon[35327]: pgmap v501: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:11.023 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:10 smithi038 ceph-mon[31227]: pgmap v501: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:11.053 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dfdb7625', total seconds elapsed: 33.2 2024-09-19T05:46:11.053 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T05:46:11.053 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:46:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 -- 172.21.15.38:0/785096089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 msgr2=0x7f89a8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 --2- 172.21.15.38:0/785096089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a8100ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8990009930 tx=0x7f899002a420 comp rx=0 tx=0).stop 2024-09-19T05:46:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 -- 172.21.15.38:0/785096089 shutdown_connections 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 --2- 172.21.15.38:0/785096089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8101420 0x7f89a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 --2- 172.21.15.38:0/785096089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a8100ee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 --2- 172.21.15.38:0/785096089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8106a80 0x7f89a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 -- 172.21.15.38:0/785096089 >> 172.21.15.38:0/785096089 conn(0x7f89a80fc7a0 msgr2=0x7f89a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 -- 172.21.15.38:0/785096089 shutdown_connections 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 -- 172.21.15.38:0/785096089 wait complete. 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.810+0000 7f89ad18d640 1 Processor -- start 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 -- start start 2024-09-19T05:46:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8101420 0x7f89a8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 0x7f89a819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8110e80 con 0x7f89a8100a80 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8110d00 con 0x7f89a8101420 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89ad18d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8111000 con 0x7f89a8106a80 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59018/0 (socket says 172.21.15.38:59018) 2024-09-19T05:46:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 -- 172.21.15.38:0/2182035629 learned_addr learned my addr 172.21.15.38:0/2182035629 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f899ffff640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8101420 0x7f89a8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a759e640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 0x7f89a819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 -- 172.21.15.38:0/2182035629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 msgr2=0x7f89a819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 0x7f89a819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 -- 172.21.15.38:0/2182035629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8101420 msgr2=0x7f89a8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8101420 0x7f89a8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a6d9d640 1 -- 172.21.15.38:0/2182035629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a819d480 con 0x7f89a8100a80 2024-09-19T05:46:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.811+0000 7f89a759e640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 0x7f89a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89a6d9d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998017070 con 0x7f89a8100a80 2024-09-19T05:46:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a819d770 con 0x7f89a8100a80 2024-09-19T05:46:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8998009c30 con 0x7f89a8100a80 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899800f6b0 con 0x7f89a8100a80 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.812+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81a5050 con 0x7f89a8100a80 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.813+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8100ee0 con 0x7f89a8100a80 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.814+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f89980109d0 con 0x7f89a8100a80 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.815+0000 7f89a4d99640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 0x7f89940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.815+0000 7f899ffff640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 0x7f89940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.815+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f89980cf900 con 0x7f89a8100a80 2024-09-19T05:46:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.815+0000 7f899ffff640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 0x7f89940a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f89900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:11.819+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8998009de0 con 0x7f89a8100a80 2024-09-19T05:46:12.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.031+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89a810a000 con 0x7f89a8100a80 2024-09-19T05:46:12.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.032+0000 7f89a4d99640 1 -- 172.21.15.38:0/2182035629 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f899809c200 con 0x7f89a8100a80 2024-09-19T05:46:12.035 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:46:12.035 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:46:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:46:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.034+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 msgr2=0x7f89940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.034+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 0x7f89940a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f89900058d0 comp rx=0 tx=0).stop 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.034+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 msgr2=0x7f89a81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.034+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).stop 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 shutdown_connections 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f89940a3240 0x7f89940a5700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89a8106a80 0x7f89a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a8101420 0x7f89a8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 --2- 172.21.15.38:0/2182035629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89a8100a80 0x7f89a81984d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 >> 172.21.15.38:0/2182035629 conn(0x7f89a80fc7a0 msgr2=0x7f89a806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 shutdown_connections 2024-09-19T05:46:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:12.035+0000 7f89ad18d640 1 -- 172.21.15.38:0/2182035629 wait complete. 2024-09-19T05:46:12.896 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T05:46:12.896 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T05:46:13.157 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:12 smithi038 ceph-mon[31227]: pgmap v502: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:46:13.157 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:12 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2182035629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:13.163 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:13.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:12 smithi179 ceph-mon[35781]: pgmap v502: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:46:13.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:12 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2182035629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:13.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:12 smithi047 ceph-mon[35327]: pgmap v502: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:46:13.229 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:12 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2182035629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:13.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 -- 172.21.15.38:0/1687177894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102ef0 msgr2=0x7f25a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 --2- 172.21.15.38:0/1687177894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102ef0 0x7f25a010da40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2598009ca0 tx=0x7f259802a950 comp rx=0 tx=0).stop 2024-09-19T05:46:13.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 -- 172.21.15.38:0/1687177894 shutdown_connections 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 --2- 172.21.15.38:0/1687177894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102ef0 0x7f25a010da40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 --2- 172.21.15.38:0/1687177894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102570 0x7f25a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 --2- 172.21.15.38:0/1687177894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 0x7f25a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 -- 172.21.15.38:0/1687177894 >> 172.21.15.38:0/1687177894 conn(0x7f25a00fc7d0 msgr2=0x7f25a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 -- 172.21.15.38:0/1687177894 shutdown_connections 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.645+0000 7f25a6afa640 1 -- 172.21.15.38:0/1687177894 wait complete. 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 Processor -- start 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 -- start start 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 0x7f25a019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 0x7f25a019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0110f80 con 0x7f25a0102570 2024-09-19T05:46:13.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0110e00 con 0x7f25a0102ef0 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a6afa640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0111100 con 0x7f25a0100e20 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59050/0 (socket says 172.21.15.38:59050) 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f259ffff640 1 -- 172.21.15.38:0/3474055396 learned_addr learned my addr 172.21.15.38:0/3474055396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.646+0000 7f25a4896640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 0x7f25a019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f25a5097640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 0x7f25a019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 -- 172.21.15.38:0/3474055396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 msgr2=0x7f25a019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 0x7f25a019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 -- 172.21.15.38:0/3474055396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 msgr2=0x7f25a019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 0x7f25a019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 -- 172.21.15.38:0/3474055396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a019f600 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f25a5097640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 0x7f25a019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259ffff640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d950 tx=0x7f258c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259dffb640 1 -- 172.21.15.38:0/3474055396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c017070 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a019f8f0 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259dffb640 1 -- 172.21.15.38:0/3474055396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009c30 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f259dffb640 1 -- 172.21.15.38:0/3474055396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c00f6b0 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.647+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a01a71d0 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.650+0000 7f259dffb640 1 -- 172.21.15.38:0/3474055396 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f258c0109d0 con 0x7f25a0102570 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.650+0000 7f25a8ed3740 1 Processor -- start 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.650+0000 7f25a8ed3740 1 -- start start 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7abb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e11b7112a0 con 0x55e11b7ac080 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e11b6b8b10 con 0x55e11b7b6940 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e11b70d260 con 0x55e11b7ab760 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59058/0 (socket says 172.21.15.38:59058) 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 -- 172.21.15.38:0/3896688872 learned_addr learned my addr 172.21.15.38:0/3896688872 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a5097640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f259ffff640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 -- 172.21.15.38:0/3896688872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 msgr2=0x55e11b7abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 -- 172.21.15.38:0/3896688872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 msgr2=0x55e11b7b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 -- 172.21.15.38:0/3896688872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e11b482b90 con 0x55e11b7ac080 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a5097640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f259dffb640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 0x7f25900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a4896640 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f259400bd10 tx=0x7f25940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaabeb4d4f27ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.651+0000 7f25a5097640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 0x7f25900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f259dffb640 1 -- 172.21.15.38:0/3474055396 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f258c0cf960 con 0x7f25a0102570 2024-09-19T05:46:13.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f258affd640 1 -- 172.21.15.38:0/3896688872 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594004030 con 0x55e11b7ac080 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f258affd640 1 -- 172.21.15.38:0/3896688872 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25940041d0 con 0x55e11b7ac080 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f258affd640 1 -- 172.21.15.38:0/3896688872 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594011840 con 0x55e11b7ac080 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3896688872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 msgr2=0x55e11b7b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a5097640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 0x7f25900a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2598006010 tx=0x7f2598005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f259400bd10 tx=0x7f25940107a0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3896688872 shutdown_connections 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b6400 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3896688872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.652+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3896688872 >> 172.21.15.38:0/3896688872 conn(0x55e11b7a7cf0 msgr2=0x55e11b7a80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3896688872 shutdown_connections 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3896688872 wait complete. 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 -- start start 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 Processor -- start 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.653+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e11b7b9510 con 0x55e11b7ac080 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e11b6996e0 con 0x55e11b7b6940 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e11b7b9690 con 0x55e11b7ab760 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a8ed3740 10 client.0 _ll_register_callbacks cb 0x55e11b6bda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59060/0 (socket says 172.21.15.38:59060) 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 -- 172.21.15.38:0/3285096515 learned_addr learned my addr 172.21.15.38:0/3285096515 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a4896640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a5097640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 -- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 msgr2=0x55e11b7b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 -- 172.21.15.38:0/3285096515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 msgr2=0x55e11b7b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 -- 172.21.15.38:0/3285096515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11b7c2a00 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f259ffff640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f258c09bb50 tx=0x7f258c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e3f329a7532b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f257ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c020070 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25940041d0 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.654+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f258c0860b0 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f258c09c2c0 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e11b7c53c0 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25940041d0 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f258c09b010 con 0x55e11b7ac080 2024-09-19T05:46:13.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 10 client.15690 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 20 client.15690 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b7c6880 0x55e11b7c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 4 client.15690 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e11b71e9d0 con 0x55e11b7c6880 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.655+0000 7f25a4896640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b7c6880 0x55e11b7c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.656+0000 7f25a4896640 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b7c6880 0x55e11b7c8c70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc41ec7d6718d96 server_cookie=f65b20ead83393c4 in_seq=0 out_seq=0 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.656+0000 7f257ffff640 10 client.15690 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.656+0000 7f257ffff640 1 -- 172.21.15.38:0/3285096515 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e11b71e9d0 con 0x55e11b7c6880 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.656+0000 7f257ffff640 10 client.15690 handle_command_reply: tid=0 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.656+0000 7f25a8ed3740 1 client.15690 shutdown 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 msgr2=0x55e11b7b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f258c09bb50 tx=0x7f258c005880 comp rx=0 tx=0).stop 2024-09-19T05:46:13.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 shutdown_connections 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b7c6880 0x55e11b7c8c70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7b6940 0x55e11b7b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7b3d50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/3285096515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7ab760 0x55e11b7b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f257ffff640 0 client.15690 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 >> 172.21.15.38:0/3285096515 conn(0x55e11b7a7cf0 msgr2=0x55e11b7ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 shutdown_connections 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 1 -- 172.21.15.38:0/3285096515 wait complete. 2024-09-19T05:46:13.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.657+0000 7f25a8ed3740 20 client.15690 trim_cache size 0 max 16384 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 Processor -- start 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 -- start start 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 0x55e11b7b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b7cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01b0e70 con 0x55e11b7ac080 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01b29b0 con 0x55e11b7ab760 2024-09-19T05:46:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01b1540 con 0x55e11b7b6940 2024-09-19T05:46:13.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f259ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59074/0 (socket says 172.21.15.38:59074) 2024-09-19T05:46:13.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f259ffff640 1 -- 172.21.15.38:0/554544891 learned_addr learned my addr 172.21.15.38:0/554544891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a4896640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 0x55e11b7b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f25a5097640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b7cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f259ffff640 1 -- 172.21.15.38:0/554544891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 msgr2=0x55e11b7cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.674+0000 7f259ffff640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b7cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f259ffff640 1 -- 172.21.15.38:0/554544891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 msgr2=0x55e11b7b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f259ffff640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 0x55e11b7b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f259ffff640 1 -- 172.21.15.38:0/554544891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e11b7a7b20 con 0x55e11b7ac080 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f25a4896640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 0x55e11b7b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f259ffff640 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f258c098590 tx=0x7f258c09a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5aca4925c38e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f258a7fc640 1 -- 172.21.15.38:0/554544891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c020070 con 0x55e11b7ac080 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f258a7fc640 1 -- 172.21.15.38:0/554544891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c09bbc0 con 0x55e11b7ac080 2024-09-19T05:46:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.675+0000 7f258a7fc640 1 -- 172.21.15.38:0/554544891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c09ac10 con 0x55e11b7ac080 2024-09-19T05:46:13.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- 172.21.15.38:0/554544891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 msgr2=0x55e11b7ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f258c098590 tx=0x7f258c09a0d0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- 172.21.15.38:0/554544891 shutdown_connections 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b7cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ac080 0x55e11b7ce4f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/554544891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ab760 0x55e11b7b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- 172.21.15.38:0/554544891 >> 172.21.15.38:0/554544891 conn(0x55e11b7a7cf0 msgr2=0x55e11b7b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- 172.21.15.38:0/554544891 shutdown_connections 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- 172.21.15.38:0/554544891 wait complete. 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 -- start start 2024-09-19T05:46:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.676+0000 7f25a8ed3740 1 Processor -- start 2024-09-19T05:46:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ac080 0x55e11b7baed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b8304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01b22e0 con 0x55e11b7ab760 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01b1c10 con 0x55e11b7ac080 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25a01aee50 con 0x55e11b7b6940 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 10 client.0 _ll_register_callbacks cb 0x7f258c09b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59082/0 (socket says 172.21.15.38:59082) 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 -- 172.21.15.38:0/1651089830 learned_addr learned my addr 172.21.15.38:0/1651089830 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a5097640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b8304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f259ffff640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ac080 0x55e11b7baed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 -- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 msgr2=0x55e11b8304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b8304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 -- 172.21.15.38:0/1651089830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ac080 msgr2=0x55e11b7baed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ac080 0x55e11b7baed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 -- 172.21.15.38:0/1651089830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11b7bb780 con 0x55e11b7ab760 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a4896640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f259400bd10 tx=0x7f259400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad256d13b3a540ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f257ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594005240 con 0x55e11b7ab760 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f25940053e0 con 0x55e11b7ab760 2024-09-19T05:46:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.677+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f258c09bbc0 con 0x55e11b7ab760 2024-09-19T05:46:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2594039050 con 0x55e11b7ab760 2024-09-19T05:46:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e11b831ac0 con 0x55e11b7ab760 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c09bbc0 con 0x55e11b7ab760 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2594030630 con 0x55e11b7ab760 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 10 client.15696 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 20 client.15696 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b831f80 0x55e11b834370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 4 client.15696 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x55e11b718d20 con 0x55e11b831f80 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.678+0000 7f259ffff640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b831f80 0x55e11b834370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.679+0000 7f259ffff640 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b831f80 0x55e11b834370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b531f714f1f3d86 server_cookie=371dd8743466335 in_seq=0 out_seq=0 2024-09-19T05:46:13.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.679+0000 7f257ffff640 10 client.15696 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:13.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f257ffff640 1 -- 172.21.15.38:0/1651089830 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e11b718d20 con 0x55e11b831f80 2024-09-19T05:46:13.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f257ffff640 10 client.15696 handle_command_reply: tid=0 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 client.15696 shutdown 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 msgr2=0x55e11b7ba990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f259400bd10 tx=0x7f259400b970 comp rx=0 tx=0).stop 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 shutdown_connections 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55e11b831f80 0x55e11b834370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e11b7b6940 0x55e11b8304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11b7ac080 0x55e11b7baed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 --2- 172.21.15.38:0/1651089830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e11b7ab760 0x55e11b7ba990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 >> 172.21.15.38:0/1651089830 conn(0x55e11b7a7cf0 msgr2=0x55e11b7a6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.688+0000 7f257ffff640 0 client.15696 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 shutdown_connections 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a8ed3740 1 -- 172.21.15.38:0/1651089830 wait complete. 2024-09-19T05:46:13.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a8ed3740 20 client.15696 trim_cache size 0 max 16384 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 22, 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "91c96d95": { 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 22, 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:46:13.707 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.8, 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:13.708 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:13.709 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:13.709 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:46:13.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 msgr2=0x7f25900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 0x7f25900a5700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2598006010 tx=0x7f2598005d80 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 msgr2=0x7f25a019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d950 tx=0x7f258c00de10 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 shutdown_connections 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f25900a3240 0x7f25900a5700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25a0102ef0 0x7f25a019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25a0102570 0x7f25a019ab90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 --2- 172.21.15.38:0/3474055396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a0100e20 0x7f25a019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 >> 172.21.15.38:0/3474055396 conn(0x7f25a00fc7d0 msgr2=0x7f25a006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.689+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 shutdown_connections 2024-09-19T05:46:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:13.690+0000 7f25a6afa640 1 -- 172.21.15.38:0/3474055396 wait complete. 2024-09-19T05:46:14.503 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '91c96d95', quiesce took 3.5 seconds. Will release after: 26.3 2024-09-19T05:46:15.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:14 smithi038 ceph-mon[31227]: pgmap v503: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:14 smithi179 ceph-mon[35781]: pgmap v503: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:14 smithi047 ceph-mon[35327]: pgmap v503: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:17.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:16 smithi038 ceph-mon[31227]: pgmap v504: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:17.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:16 smithi179 ceph-mon[35781]: pgmap v504: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:17.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:16 smithi047 ceph-mon[35327]: pgmap v504: 129 pgs: 129 active+clean; 2.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:46:19.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:18 smithi038 ceph-mon[31227]: pgmap v505: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:46:19.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:18 smithi179 ceph-mon[35781]: pgmap v505: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:46:19.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:18 smithi047 ceph-mon[35327]: pgmap v505: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:46:21.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:20 smithi038 ceph-mon[31227]: pgmap v506: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:46:21.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:20 smithi179 ceph-mon[35781]: pgmap v506: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:46:21.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:20 smithi047 ceph-mon[35327]: pgmap v506: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:46:23.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:22 smithi038 ceph-mon[31227]: pgmap v507: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:46:23.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:22 smithi179 ceph-mon[35781]: pgmap v507: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:46:23.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:22 smithi047 ceph-mon[35327]: pgmap v507: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:46:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:24 smithi038 ceph-mon[31227]: pgmap v508: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:24 smithi179 ceph-mon[35781]: pgmap v508: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:25.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:24 smithi047 ceph-mon[35327]: pgmap v508: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:27.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:26 smithi038 ceph-mon[31227]: pgmap v509: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:26 smithi179 ceph-mon[35781]: pgmap v509: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:27.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:26 smithi047 ceph-mon[35327]: pgmap v509: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:46:29.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:28 smithi038 ceph-mon[31227]: pgmap v510: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T05:46:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:28 smithi179 ceph-mon[35781]: pgmap v510: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T05:46:29.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:28 smithi047 ceph-mon[35327]: pgmap v510: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T05:46:31.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:30 smithi038 ceph-mon[31227]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:46:31.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:30 smithi179 ceph-mon[35781]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:46:31.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:30 smithi047 ceph-mon[35327]: pgmap v511: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:46:33.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:32 smithi038 ceph-mon[31227]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:46:33.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:32 smithi179 ceph-mon[35781]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:46:33.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:32 smithi047 ceph-mon[35327]: pgmap v512: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:46:35.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:34 smithi038 ceph-mon[31227]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:35.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:34 smithi179 ceph-mon[35781]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:34 smithi047 ceph-mon[35327]: pgmap v513: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:37.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:36 smithi038 ceph-mon[31227]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:37.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:36 smithi179 ceph-mon[35781]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:37.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:36 smithi047 ceph-mon[35327]: pgmap v514: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:39.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:38 smithi038 ceph-mon[31227]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:38 smithi179 ceph-mon[35781]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:39.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:38 smithi047 ceph-mon[35327]: pgmap v515: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:40.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:46:41.082 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:41.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:40 smithi038 ceph-mon[31227]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:41.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:40 smithi179 ceph-mon[35781]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:40 smithi047 ceph-mon[35327]: pgmap v516: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 -- 172.21.15.38:0/2238068270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 msgr2=0x7fd0a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 --2- 172.21.15.38:0/2238068270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a010da40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd098009cc0 tx=0x7fd09802cb60 comp rx=0 tx=0).stop 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 -- 172.21.15.38:0/2238068270 shutdown_connections 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 --2- 172.21.15.38:0/2238068270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a010da40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 --2- 172.21.15.38:0/2238068270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 0x7fd0a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 --2- 172.21.15.38:0/2238068270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 -- 172.21.15.38:0/2238068270 >> 172.21.15.38:0/2238068270 conn(0x7fd0a00fc7d0 msgr2=0x7fd0a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 -- 172.21.15.38:0/2238068270 shutdown_connections 2024-09-19T05:46:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.606+0000 7fd0a784b640 1 -- 172.21.15.38:0/2238068270 wait complete. 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 Processor -- start 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 -- start start 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 0x7fd0a0196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010fe80 con 0x7fd0a0102ef0 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010fd00 con 0x7fd0a0102570 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a784b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0110000 con 0x7fd0a0100e20 2024-09-19T05:46:41.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.607+0000 7fd0a5de8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46654/0 (socket says 172.21.15.38:46654) 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 -- 172.21.15.38:0/1962264881 learned_addr learned my addr 172.21.15.38:0/1962264881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a4de6640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 0x7fd0a0196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a55e7640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 -- 172.21.15.38:0/1962264881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 msgr2=0x7fd0a0196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 -- 172.21.15.38:0/1962264881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 msgr2=0x7fd0a0196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:41.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 0x7fd0a0196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 -- 172.21.15.38:0/1962264881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a019b1b0 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a55e7640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a5de8640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd098006010 tx=0x7fd098003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098044070 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a019b440 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098003e50 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098036910 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.608+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a2d80 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.609+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0101200 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.612+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd098047070 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.612+0000 7fd0967fc640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 0x7fd08c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.612+0000 7fd0a55e7640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 0x7fd08c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.613+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fd0980f2f50 con 0x7fd0a0102ef0 2024-09-19T05:46:41.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.613+0000 7fd0a55e7640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 0x7fd08c0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00abe0 tx=0x7fd09c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:41.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.615+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0980bf8b0 con 0x7fd0a0102ef0 2024-09-19T05:46:41.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.829+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a01a2fb0 con 0x7fd0a0102ef0 2024-09-19T05:46:41.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.829+0000 7fd0967fc640 1 -- 172.21.15.38:0/1962264881 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fd0980bf1b0 con 0x7fd0a0102ef0 2024-09-19T05:46:41.833 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:46:41.833 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:46:41.833 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:46:41.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 msgr2=0x7fd08c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:41.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 0x7fd08c0a5700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00abe0 tx=0x7fd09c00a5c0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 msgr2=0x7fd0a019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd098006010 tx=0x7fd098003ce0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 shutdown_connections 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd08c0a3240 0x7fd08c0a5700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a0102ef0 0x7fd0a019aad0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0a0102570 0x7fd0a0196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 --2- 172.21.15.38:0/1962264881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd0a0100e20 0x7fd0a0196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:41.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.832+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 >> 172.21.15.38:0/1962264881 conn(0x7fd0a00fc7d0 msgr2=0x7fd0a01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:41.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.833+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 shutdown_connections 2024-09-19T05:46:41.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:41.833+0000 7fd0a784b640 1 -- 172.21.15.38:0/1962264881 wait complete. 2024-09-19T05:46:42.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:41 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1962264881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:42.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:41 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1962264881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:42.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:41 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1962264881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:46:42.630 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 91c96d95 --release --await' 2024-09-19T05:46:42.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 91c96d95 --release --await 2024-09-19T05:46:42.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:42 smithi038 ceph-mon[31227]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:42.897 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:46:43.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:42 smithi179 ceph-mon[35781]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:43.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:42 smithi047 ceph-mon[35327]: pgmap v517: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 -- 172.21.15.38:0/1688531699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8106f20 msgr2=0x7f99c810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 --2- 172.21.15.38:0/1688531699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8106f20 0x7f99c810eaa0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009ca0 tx=0x7f99c002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 -- 172.21.15.38:0/1688531699 shutdown_connections 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 --2- 172.21.15.38:0/1688531699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8106f20 0x7f99c810eaa0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 --2- 172.21.15.38:0/1688531699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 0x7f99c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 --2- 172.21.15.38:0/1688531699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8101df0 0x7f99c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 -- 172.21.15.38:0/1688531699 >> 172.21.15.38:0/1688531699 conn(0x7f99c80fd8f0 msgr2=0x7f99c80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 -- 172.21.15.38:0/1688531699 shutdown_connections 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 -- 172.21.15.38:0/1688531699 wait complete. 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.364+0000 7f99cccef640 1 Processor -- start 2024-09-19T05:46:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 -- start start 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 0x7f99c8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8106f20 0x7f99c819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8111fa0 con 0x7f99c8101df0 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8111e20 con 0x7f99c81027a0 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99cccef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8112120 con 0x7f99c8106f20 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46674/0 (socket says 172.21.15.38:46674) 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 -- 172.21.15.38:0/4087453038 learned_addr learned my addr 172.21.15.38:0/4087453038 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 0x7f99c8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8106f20 0x7f99c819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 -- 172.21.15.38:0/4087453038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8106f20 msgr2=0x7f99c819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8106f20 0x7f99c819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 -- 172.21.15.38:0/4087453038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 msgr2=0x7f99c8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 0x7f99c8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.365+0000 7f99c659c640 1 -- 172.21.15.38:0/4087453038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c819d3f0 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99c659c640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4a0 tx=0x7f99b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99bffff640 1 -- 172.21.15.38:0/4087453038 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8014070 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99bffff640 1 -- 172.21.15.38:0/4087453038 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b8010af0 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c819d6e0 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99bffff640 1 -- 172.21.15.38:0/4087453038 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017d10 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.366+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a4fc0 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.368+0000 7f99bffff640 1 -- 172.21.15.38:0/4087453038 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99c8101df0 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.368+0000 7f99cf0c8740 1 Processor -- start 2024-09-19T05:46:43.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.368+0000 7f99cf0c8740 1 -- start start 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30acc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30adda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e30a31620 con 0x563e30acc880 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e309d8e50 con 0x563e30acbf60 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e30a2d5e0 con 0x563e30adb650 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46690/0 (socket says 172.21.15.38:46690) 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 -- 172.21.15.38:0/247082812 learned_addr learned my addr 172.21.15.38:0/247082812 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30acc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99bffff640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 0x7f99b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30adda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 0x7f99b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99bffff640 1 -- 172.21.15.38:0/4087453038 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f99b80cf1c0 con 0x7f99c8101df0 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 -- 172.21.15.38:0/247082812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 msgr2=0x563e30adda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30adda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 -- 172.21.15.38:0/247082812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 msgr2=0x563e30acc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30acc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c659c640 1 -- 172.21.15.38:0/247082812 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e309c9d40 con 0x563e30acc880 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.369+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30acc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99c659c640 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f99b809b120 tx=0x7f99b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c669073f1b95cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99bcd99640 1 -- 172.21.15.38:0/247082812 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b801e070 con 0x563e30acc880 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99bcd99640 1 -- 172.21.15.38:0/247082812 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b809f7d0 con 0x563e30acc880 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99bcd99640 1 -- 172.21.15.38:0/247082812 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8097b10 con 0x563e30acc880 2024-09-19T05:46:43.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 0x7f99b40a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009430 tx=0x7f99c002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 -- 172.21.15.38:0/247082812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 msgr2=0x563e30adb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f99b809b120 tx=0x7f99b80046c0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 -- 172.21.15.38:0/247082812 shutdown_connections 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30adda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30adb110 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/247082812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30acc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.370+0000 7f99cf0c8740 1 -- 172.21.15.38:0/247082812 >> 172.21.15.38:0/247082812 conn(0x563e30ac85f0 msgr2=0x563e30ac89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- 172.21.15.38:0/247082812 shutdown_connections 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- 172.21.15.38:0/247082812 wait complete. 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- start start 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 Processor -- start 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30ad8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30ad9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563e30ade220 con 0x563e30acc880 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e309b9f10 con 0x563e30acbf60 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e30ade3a0 con 0x563e30adb650 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99cf0c8740 10 client.0 _ll_register_callbacks cb 0x563e309dddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99bdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99bdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46706/0 (socket says 172.21.15.38:46706) 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.371+0000 7f99bdd9b640 1 -- 172.21.15.38:0/459693796 learned_addr learned my addr 172.21.15.38:0/459693796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99c659c640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30ad8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30ad9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 -- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 msgr2=0x563e30ad9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30ad9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 -- 172.21.15.38:0/459693796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 msgr2=0x563e30ad8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30ad8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 -- 172.21.15.38:0/459693796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e30740580 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f99b0012700 tx=0x7f99b0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98db0059407e4eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99adffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0004320 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f99b00119a0 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b809f7d0 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b00044c0 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99cf0c8740 10 client.15714 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3080a960 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.372+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b809f7d0 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99b000f440 con 0x563e30acc880 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 10 client.15714 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 20 client.15714 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30aeb320 0x563e30aed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 4 client.15714 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e30a3eb90 con 0x563e30aeb320 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99c659c640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30aeb320 0x563e30aed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99c659c640 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30aeb320 0x563e30aed770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8484a4dc2ee2ec4 server_cookie=dad2a96c0d9aae2a in_seq=0 out_seq=0 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.373+0000 7f99adffb640 10 client.15714 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99adffb640 1 -- 172.21.15.38:0/459693796 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e30a3eb90 con 0x563e30aeb320 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99adffb640 10 client.15714 handle_command_reply: tid=0 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 client.15714 shutdown 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 msgr2=0x563e30ad8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f99b0012700 tx=0x7f99b0012bc0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 shutdown_connections 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30aeb320 0x563e30aed770 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30ad9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acc880 0x563e30ad8b40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/459693796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acbf60 0x563e30ad8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99adffb640 0 client.15714 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 >> 172.21.15.38:0/459693796 conn(0x563e30ac85f0 msgr2=0x563e30ace350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 shutdown_connections 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.374+0000 7f99cf0c8740 1 -- 172.21.15.38:0/459693796 wait complete. 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.375+0000 7f99cf0c8740 20 client.15714 trim_cache size 0 max 16384 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 Processor -- start 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 -- start start 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30ada570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30af3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81acc20 con 0x563e30acbf60 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81ae760 con 0x563e30acc880 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81ad2f0 con 0x563e30adb650 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46714/0 (socket says 172.21.15.38:46714) 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 -- 172.21.15.38:0/907872150 learned_addr learned my addr 172.21.15.38:0/907872150 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30ada570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30af3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 -- 172.21.15.38:0/907872150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 msgr2=0x563e30af3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30af3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 -- 172.21.15.38:0/907872150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 msgr2=0x563e30ada570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30ada570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99c659c640 1 -- 172.21.15.38:0/907872150 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e30ad8ca0 con 0x563e30acbf60 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.392+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30ada570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99c659c640 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f99b8097880 tx=0x7f99b800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aefa908ece203a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99affff640 1 -- 172.21.15.38:0/907872150 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b801e040 con 0x563e30acbf60 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99affff640 1 -- 172.21.15.38:0/907872150 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b809f7d0 con 0x563e30acbf60 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99affff640 1 -- 172.21.15.38:0/907872150 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8097a90 con 0x563e30acbf60 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99cf0c8740 1 -- 172.21.15.38:0/907872150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 msgr2=0x563e30ad3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.393+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f99b8097880 tx=0x7f99b800bf90 comp rx=0 tx=0).stop 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- 172.21.15.38:0/907872150 shutdown_connections 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30af3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30ada570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/907872150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30ad3b00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- 172.21.15.38:0/907872150 >> 172.21.15.38:0/907872150 conn(0x563e30ac85f0 msgr2=0x563e30ad9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- 172.21.15.38:0/907872150 shutdown_connections 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- 172.21.15.38:0/907872150 wait complete. 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- start start 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 Processor -- start 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30adfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30b55010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81ae090 con 0x563e30acbf60 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81ad9c0 con 0x563e30acc880 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99c81aac00 con 0x563e30adb650 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99cf0c8740 10 client.0 _ll_register_callbacks cb 0x7f99b0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99c659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46724/0 (socket says 172.21.15.38:46724) 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99c659c640 1 -- 172.21.15.38:0/4035155290 learned_addr learned my addr 172.21.15.38:0/4035155290 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.394+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30adfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c6d9d640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30b55010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 -- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 msgr2=0x563e30b55010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30b55010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 -- 172.21.15.38:0/4035155290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 msgr2=0x563e30adfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30adfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 -- 172.21.15.38:0/4035155290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e30b55550 con 0x563e30acbf60 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99c659c640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f99b80a3b60 tx=0x7f99b80a4210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4b84fe70e477e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99adffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b801e070 con 0x563e30acbf60 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f99b8099720 con 0x563e30acbf60 2024-09-19T05:46:43.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b809f7d0 con 0x563e30acbf60 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 24, 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "91c96d95": { 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 24, 2024-09-19T05:46:43.491 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.8, 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:46:43.492 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99b8017360 con 0x563e30acbf60 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99cf0c8740 10 client.15720 fetch_fsmap learned FSMap version 28 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e30b56750 con 0x563e30acbf60 2024-09-19T05:46:43.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.395+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b809f7d0 con 0x563e30acbf60 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f99b80b40d0 con 0x563e30acbf60 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 10 client.15720 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 20 client.15720 populate_metadata read hostname 'smithi038' 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30b56920 0x563e30b58d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 4 client.15720 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "91c96d95", "release": true, "await": true}] 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "91c96d95", "release": true, "await": true}) -- 0x563e30ad9410 con 0x563e30b56920 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30b56920 0x563e30b58d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99bdd9b640 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30b56920 0x563e30b58d10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d543493c13f04a96 server_cookie=2b888c89f563edd8 in_seq=0 out_seq=0 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.396+0000 7f99adffb640 10 client.15720 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99adffb640 1 -- 172.21.15.38:0/4035155290 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563e30ad9410 con 0x563e30b56920 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99adffb640 10 client.15720 handle_command_reply: tid=0 2024-09-19T05:46:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 client.15720 shutdown 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 msgr2=0x563e30adf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f99b80a3b60 tx=0x7f99b80a4210 comp rx=0 tx=0).stop 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 shutdown_connections 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x563e30b56920 0x563e30b58d10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e30adb650 0x563e30b55010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e30acc880 0x563e30adfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99adffb640 0 client.15720 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 --2- 172.21.15.38:0/4035155290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563e30acbf60 0x563e30adf7b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 >> 172.21.15.38:0/4035155290 conn(0x563e30ac85f0 msgr2=0x563e30ac7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.399+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 shutdown_connections 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cf0c8740 1 -- 172.21.15.38:0/4035155290 wait complete. 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cf0c8740 20 client.15720 trim_cache size 0 max 16384 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 msgr2=0x7f99b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 0x7f99b40a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009430 tx=0x7f99c002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 msgr2=0x7f99c8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4a0 tx=0x7f99b800b960 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 shutdown_connections 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f99b40a3240 0x7f99b40a5700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99c8106f20 0x7f99c819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c81027a0 0x7f99c8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 --2- 172.21.15.38:0/4087453038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99c8101df0 0x7f99c8198440 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 >> 172.21.15.38:0/4087453038 conn(0x7f99c80fd8f0 msgr2=0x7f99c80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 shutdown_connections 2024-09-19T05:46:43.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:46:43.400+0000 7f99cccef640 1 -- 172.21.15.38:0/4087453038 wait complete. 2024-09-19T05:46:44.219 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '91c96d95', total seconds elapsed: 33.2 2024-09-19T05:46:44.408 INFO:tasks.workunit.client.0.smithi038.stdout:64/600: open fd = 3 2024-09-19T05:46:44.408 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.408 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:65/600: open fd = 3 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:66/600: open fd = 3 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:67/600: open fd = 3 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.409 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:68/600: open fd = 3 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:69/600: open fd = 3 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:70/600: open fd = 3 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.410 INFO:tasks.workunit.client.0.smithi038.stdout:71/600: open fd = 3 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:72/600: open fd = 3 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:73/600: open fd = 3 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:74/600: open fd = 3 2024-09-19T05:46:44.411 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:75/600: open fd = 3 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:76/600: open fd = 3 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:77/600: open fd = 3 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.412 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:78/600: open fd = 3 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:79/600: open fd = 3 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:80/600: open fd = 3 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.413 INFO:tasks.workunit.client.0.smithi038.stdout:81/600: open fd = 3 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:82/600: open fd = 3 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:83/600: open fd = 3 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.414 INFO:tasks.workunit.client.0.smithi038.stdout:84/600: open fd = 3 2024-09-19T05:46:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:85/600: open fd = 3 2024-09-19T05:46:44.665 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:86/600: open fd = 3 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:87/600: open fd = 3 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:88/600: open fd = 3 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.666 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:89/600: open fd = 3 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:90/600: open fd = 3 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:91/600: open fd = 3 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.667 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:92/600: open fd = 3 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:93/600: open fd = 3 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:94/600: open fd = 3 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.668 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:95/600: open fd = 3 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:96/600: open fd = 3 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:97/600: open fd = 3 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.669 INFO:tasks.workunit.client.0.smithi038.stdout:98/600: open fd = 3 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:99/600: open fd = 3 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:100/600: open fd = 3 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:101/600: open fd = 3 2024-09-19T05:46:44.670 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:102/600: open fd = 3 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:103/600: open fd = 3 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:104/600: open fd = 3 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.671 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:105/600: open fd = 3 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:106/600: open fd = 3 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:107/600: open fd = 3 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.672 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:108/600: open fd = 3 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:109/600: open fd = 3 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:110/600: open fd = 3 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.673 INFO:tasks.workunit.client.0.smithi038.stdout:111/600: open fd = 3 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:112/600: open fd = 3 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:113/600: open fd = 3 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:114/600: open fd = 3 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.674 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:115/600: open fd = 3 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:116/600: open fd = 3 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:117/600: open fd = 3 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.675 INFO:tasks.workunit.client.0.smithi038.stdout:118/600: open fd = 3 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:119/600: open fd = 3 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:120/600: open fd = 3 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.676 INFO:tasks.workunit.client.0.smithi038.stdout:121/600: open fd = 3 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:122/600: open fd = 3 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:123/600: open fd = 3 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:124/600: open fd = 3 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.677 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:125/600: open fd = 3 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:126/600: open fd = 3 2024-09-19T05:46:44.678 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:46:45.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:44 smithi038 ceph-mon[31227]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:44 smithi179 ceph-mon[35781]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:45.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:44 smithi047 ceph-mon[35327]: pgmap v518: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:46:47.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:46 smithi038 ceph-mon[31227]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:46 smithi179 ceph-mon[35781]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:46 smithi047 ceph-mon[35327]: pgmap v519: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:46:49.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:48 smithi038 ceph-mon[31227]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-19T05:46:49.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:48 smithi179 ceph-mon[35781]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-19T05:46:49.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:48 smithi047 ceph-mon[35327]: pgmap v520: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-19T05:46:51.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:50 smithi038 ceph-mon[31227]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-19T05:46:51.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:50 smithi179 ceph-mon[35781]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-19T05:46:51.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:50 smithi047 ceph-mon[35327]: pgmap v521: 129 pgs: 129 active+clean; 2.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-19T05:46:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:52 smithi038 ceph-mon[31227]: pgmap v522: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:46:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:52 smithi179 ceph-mon[35781]: pgmap v522: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:46:53.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:52 smithi047 ceph-mon[35327]: pgmap v522: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-19T05:46:55.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:54 smithi038 ceph-mon[31227]: pgmap v523: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:55.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:54 smithi179 ceph-mon[35781]: pgmap v523: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:55.237 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:54 smithi047 ceph-mon[35327]: pgmap v523: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:57.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:56 smithi179 ceph-mon[35781]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:57.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:56 smithi047 ceph-mon[35327]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:57.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:56 smithi038 ceph-mon[31227]: pgmap v524: 129 pgs: 129 active+clean; 2.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-19T05:46:59.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:46:58 smithi179 ceph-mon[35781]: pgmap v525: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T05:46:59.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:46:58 smithi047 ceph-mon[35327]: pgmap v525: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T05:46:59.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:46:58 smithi038 ceph-mon[31227]: pgmap v525: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-19T05:47:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:00 smithi179 ceph-mon[35781]: pgmap v526: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:01.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:00 smithi047 ceph-mon[35327]: pgmap v526: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:01.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:00 smithi038 ceph-mon[31227]: pgmap v526: 129 pgs: 129 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:03.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:02 smithi179 ceph-mon[35781]: pgmap v527: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:03.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:02 smithi047 ceph-mon[35327]: pgmap v527: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:03.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:02 smithi038 ceph-mon[31227]: pgmap v527: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:05.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:04 smithi179 ceph-mon[35781]: pgmap v528: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:05.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:04 smithi047 ceph-mon[35327]: pgmap v528: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:05.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:04 smithi038 ceph-mon[31227]: pgmap v528: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:06 smithi179 ceph-mon[35781]: pgmap v529: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:07.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:06 smithi047 ceph-mon[35327]: pgmap v529: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:07.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:06 smithi038 ceph-mon[31227]: pgmap v529: 129 pgs: 129 active+clean; 2.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:08.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:07 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:47:08.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:07 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:47:08.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:07 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:47:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:08 smithi179 ceph-mon[35781]: pgmap v530: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:47:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:47:09.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:08 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:47:09.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:08 smithi047 ceph-mon[35327]: pgmap v530: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:09.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:47:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:47:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:08 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:47:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:08 smithi038 ceph-mon[31227]: pgmap v530: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:47:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:47:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:08 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:47:11.210 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:10 smithi038 ceph-mon[31227]: pgmap v531: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:11.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:10 smithi047 ceph-mon[35327]: pgmap v531: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:10 smithi179 ceph-mon[35781]: pgmap v531: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:13.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:12 smithi047 ceph-mon[35327]: pgmap v532: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:12 smithi038 ceph-mon[31227]: pgmap v532: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:13.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:12 smithi179 ceph-mon[35781]: pgmap v532: 129 pgs: 129 active+clean; 2.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:14 smithi047 ceph-mon[35327]: pgmap v533: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:14 smithi038 ceph-mon[31227]: pgmap v533: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:14 smithi179 ceph-mon[35781]: pgmap v533: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:17.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:16 smithi047 ceph-mon[35327]: pgmap v534: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:16 smithi038 ceph-mon[31227]: pgmap v534: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:16 smithi179 ceph-mon[35781]: pgmap v534: 129 pgs: 129 active+clean; 2.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:47:19.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:18 smithi047 ceph-mon[35327]: pgmap v535: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:18 smithi038 ceph-mon[31227]: pgmap v535: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:18 smithi179 ceph-mon[35781]: pgmap v535: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:21.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:20 smithi047 ceph-mon[35327]: pgmap v536: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:21.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:20 smithi038 ceph-mon[31227]: pgmap v536: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:20 smithi179 ceph-mon[35781]: pgmap v536: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:47:23.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:22 smithi038 ceph-mon[31227]: pgmap v537: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:23.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:22 smithi179 ceph-mon[35781]: pgmap v537: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:23.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:22 smithi047 ceph-mon[35327]: pgmap v537: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:47:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:25 smithi179 ceph-mon[35781]: pgmap v538: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:25 smithi047 ceph-mon[35327]: pgmap v538: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:25 smithi038 ceph-mon[31227]: pgmap v538: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:27 smithi179 ceph-mon[35781]: pgmap v539: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:27.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:27 smithi047 ceph-mon[35327]: pgmap v539: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:27.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:27 smithi038 ceph-mon[31227]: pgmap v539: 129 pgs: 129 active+clean; 2.5 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:29.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:29 smithi047 ceph-mon[35327]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:29.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:29 smithi038 ceph-mon[31227]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:29 smithi179 ceph-mon[35781]: pgmap v540: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:31 smithi047 ceph-mon[35327]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:31.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:31 smithi038 ceph-mon[31227]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:31 smithi179 ceph-mon[35781]: pgmap v541: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:33.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:33 smithi047 ceph-mon[35327]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:33.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:33 smithi038 ceph-mon[31227]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:33 smithi179 ceph-mon[35781]: pgmap v542: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:35.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:35 smithi047 ceph-mon[35327]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:35 smithi038 ceph-mon[31227]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:35 smithi179 ceph-mon[35781]: pgmap v543: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:37 smithi038 ceph-mon[31227]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:37 smithi179 ceph-mon[35781]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:37.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:37 smithi047 ceph-mon[35327]: pgmap v544: 129 pgs: 129 active+clean; 2.6 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:39 smithi038 ceph-mon[31227]: pgmap v545: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:39 smithi179 ceph-mon[35781]: pgmap v545: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:39.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:39 smithi047 ceph-mon[35327]: pgmap v545: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:41.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:41 smithi038 ceph-mon[31227]: pgmap v546: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:41 smithi179 ceph-mon[35781]: pgmap v546: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:41.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:41 smithi047 ceph-mon[35327]: pgmap v546: 129 pgs: 129 active+clean; 2.7 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:43 smithi038 ceph-mon[31227]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:43 smithi179 ceph-mon[35781]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:43.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:43 smithi047 ceph-mon[35327]: pgmap v547: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:47:45.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:45 smithi038 ceph-mon[31227]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:45 smithi179 ceph-mon[35781]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:45.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:45 smithi047 ceph-mon[35327]: pgmap v548: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:46.874 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.874 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.874 INFO:tasks.workunit.client.0.smithi038.stdout:127/600: open fd = 3 2024-09-19T05:47:46.874 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.874 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:128/600: open fd = 3 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:129/600: open fd = 3 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:130/600: open fd = 3 2024-09-19T05:47:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:131/600: open fd = 3 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:132/600: open fd = 3 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:133/600: open fd = 3 2024-09-19T05:47:46.876 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:134/600: open fd = 3 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:135/600: open fd = 3 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:136/600: open fd = 3 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:137/600: open fd = 3 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:138/600: open fd = 3 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:139/600: open fd = 3 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:140/600: open fd = 3 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:141/600: open fd = 3 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:142/600: open fd = 3 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:143/600: open fd = 3 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.880 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:144/600: open fd = 3 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:145/600: open fd = 3 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:146/600: open fd = 3 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:147/600: open fd = 3 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:148/600: open fd = 3 2024-09-19T05:47:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:149/600: open fd = 3 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:150/600: open fd = 3 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:151/600: open fd = 3 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:152/600: open fd = 3 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:153/600: open fd = 3 2024-09-19T05:47:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:154/600: open fd = 3 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:155/600: open fd = 3 2024-09-19T05:47:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:156/600: open fd = 3 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:157/600: open fd = 3 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:158/600: open fd = 3 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:159/600: open fd = 3 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:160/600: open fd = 3 2024-09-19T05:47:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:161/600: open fd = 3 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:162/600: open fd = 3 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:163/600: open fd = 3 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:164/600: open fd = 3 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:165/600: open fd = 3 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:166/600: open fd = 3 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:167/600: open fd = 3 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:168/600: open fd = 3 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:169/600: open fd = 3 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:170/600: open fd = 3 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.890 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:171/600: open fd = 3 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:172/600: open fd = 3 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:173/600: open fd = 3 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:174/600: open fd = 3 2024-09-19T05:47:46.891 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:175/600: open fd = 3 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:176/600: open fd = 3 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:177/600: open fd = 3 2024-09-19T05:47:46.892 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:178/600: open fd = 3 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:179/600: open fd = 3 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:180/600: open fd = 3 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.893 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:181/600: open fd = 3 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:182/600: open fd = 3 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:183/600: open fd = 3 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:184/600: open fd = 3 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:185/600: open fd = 3 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:186/600: open fd = 3 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:187/600: open fd = 3 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:47:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:188/600: open fd = 3 2024-09-19T05:47:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:47:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:47 smithi038 ceph-mon[31227]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:47.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:47 smithi179 ceph-mon[35781]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:47.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:47 smithi047 ceph-mon[35327]: pgmap v549: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:49.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:49 smithi038 ceph-mon[31227]: pgmap v550: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:49.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:49 smithi179 ceph-mon[35781]: pgmap v550: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:49 smithi047 ceph-mon[35327]: pgmap v550: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:51.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:51 smithi038 ceph-mon[31227]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:51 smithi179 ceph-mon[35781]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:51.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:51 smithi047 ceph-mon[35327]: pgmap v551: 129 pgs: 129 active+clean; 2.8 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:53 smithi038 ceph-mon[31227]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:53 smithi179 ceph-mon[35781]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:53.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:53 smithi047 ceph-mon[35327]: pgmap v552: 129 pgs: 129 active+clean; 2.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:55.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:55 smithi038 ceph-mon[31227]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:55 smithi179 ceph-mon[35781]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:55.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:55 smithi047 ceph-mon[35327]: pgmap v553: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:47:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:57 smithi038 ceph-mon[31227]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:57 smithi179 ceph-mon[35781]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:57.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:57 smithi047 ceph-mon[35327]: pgmap v554: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:47:59.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:47:59 smithi038 ceph-mon[31227]: pgmap v555: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:47:59 smithi179 ceph-mon[35781]: pgmap v555: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:47:59.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:47:59 smithi047 ceph-mon[35327]: pgmap v555: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:01.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:01 smithi038 ceph-mon[31227]: pgmap v556: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:01 smithi179 ceph-mon[35781]: pgmap v556: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:01.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:01 smithi047 ceph-mon[35327]: pgmap v556: 129 pgs: 129 active+clean; 2.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:03.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:03 smithi038 ceph-mon[31227]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:03 smithi179 ceph-mon[35781]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:03 smithi047 ceph-mon[35327]: pgmap v557: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:05 smithi038 ceph-mon[31227]: pgmap v558: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:05 smithi179 ceph-mon[35781]: pgmap v558: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:05 smithi047 ceph-mon[35327]: pgmap v558: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:07 smithi038 ceph-mon[31227]: pgmap v559: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:07 smithi179 ceph-mon[35781]: pgmap v559: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:07 smithi047 ceph-mon[35327]: pgmap v559: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: pgmap v560: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:48:09.645 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:48:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: pgmap v560: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:48:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:48:09.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: pgmap v560: 129 pgs: 129 active+clean; 3.0 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:48:09.727 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:48:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:10 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:48:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:10 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:48:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:10 smithi038 ceph-mon[31227]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:48:10.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:10 smithi038 ceph-mon[31227]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:48:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:10 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:48:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:10 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:48:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:10 smithi179 ceph-mon[35781]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:48:10.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:10 smithi179 ceph-mon[35781]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:48:10.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:10 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:48:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:10 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:48:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:10 smithi047 ceph-mon[35327]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:48:10.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:10 smithi047 ceph-mon[35327]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:48:11.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:11 smithi038 ceph-mon[31227]: pgmap v561: 129 pgs: 129 active+clean; 3.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:11 smithi179 ceph-mon[35781]: pgmap v561: 129 pgs: 129 active+clean; 3.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:11 smithi047 ceph-mon[35327]: pgmap v561: 129 pgs: 129 active+clean; 3.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:13.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:13 smithi038 ceph-mon[31227]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:48:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:13 smithi179 ceph-mon[35781]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:48:13.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:13 smithi047 ceph-mon[35327]: pgmap v562: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:48:15.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:15 smithi038 ceph-mon[31227]: pgmap v563: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:15 smithi179 ceph-mon[35781]: pgmap v563: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:15 smithi047 ceph-mon[35327]: pgmap v563: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:17.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:17 smithi038 ceph-mon[31227]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:17 smithi179 ceph-mon[35781]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:17.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:17 smithi047 ceph-mon[35327]: pgmap v564: 129 pgs: 129 active+clean; 3.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:19 smithi038 ceph-mon[31227]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:19 smithi179 ceph-mon[35781]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:19.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:19 smithi047 ceph-mon[35327]: pgmap v565: 129 pgs: 129 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:21.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:21 smithi038 ceph-mon[31227]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:21 smithi179 ceph-mon[35781]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:21 smithi047 ceph-mon[35327]: pgmap v566: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:23.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:23 smithi038 ceph-mon[31227]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:23 smithi179 ceph-mon[35781]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:23.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:23 smithi047 ceph-mon[35327]: pgmap v567: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:25 smithi038 ceph-mon[31227]: pgmap v568: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:25 smithi179 ceph-mon[35781]: pgmap v568: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:25 smithi047 ceph-mon[35327]: pgmap v568: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:27.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:27 smithi179 ceph-mon[35781]: pgmap v569: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:27.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:27 smithi047 ceph-mon[35327]: pgmap v569: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:27 smithi038 ceph-mon[31227]: pgmap v569: 129 pgs: 129 active+clean; 3.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:29 smithi179 ceph-mon[35781]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:29.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:29 smithi047 ceph-mon[35327]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:29 smithi038 ceph-mon[31227]: pgmap v570: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:31 smithi179 ceph-mon[35781]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:31.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:31 smithi047 ceph-mon[35327]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:31 smithi038 ceph-mon[31227]: pgmap v571: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:33 smithi179 ceph-mon[35781]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:33.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:33 smithi047 ceph-mon[35327]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:33 smithi038 ceph-mon[31227]: pgmap v572: 129 pgs: 129 active+clean; 3.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:34.906 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:48:34.907 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T05:48:34.907 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:48:35.178 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:48:35.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:35 smithi038 ceph-mon[31227]: pgmap v573: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 -- 172.21.15.38:0/3275688192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968103660 msgr2=0x7f796810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 --2- 172.21.15.38:0/3275688192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968103660 0x7f796810bb50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7958009900 tx=0x7f795802a480 comp rx=0 tx=0).stop 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 -- 172.21.15.38:0/3275688192 shutdown_connections 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 --2- 172.21.15.38:0/3275688192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f796810c170 0x7f796810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 --2- 172.21.15.38:0/3275688192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968103660 0x7f796810bb50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 --2- 172.21.15.38:0/3275688192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968102d40 0x7f7968103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 -- 172.21.15.38:0/3275688192 >> 172.21.15.38:0/3275688192 conn(0x7f79680fc7a0 msgr2=0x7f79680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 -- 172.21.15.38:0/3275688192 shutdown_connections 2024-09-19T05:48:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.673+0000 7f796ef74640 1 -- 172.21.15.38:0/3275688192 wait complete. 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 Processor -- start 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 -- start start 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7968103660 0x7f7968198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796810c170 0x7f796819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7968111f70 con 0x7f7968102d40 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7968111df0 con 0x7f7968103660 2024-09-19T05:48:35.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796ef74640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79681120f0 con 0x7f796810c170 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796cd10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.674+0000 7f796cd10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57426/0 (socket says 172.21.15.38:57426) 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 -- 172.21.15.38:0/987434240 learned_addr learned my addr 172.21.15.38:0/987434240 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f795ffff640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7968103660 0x7f7968198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796d511640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796810c170 0x7f796819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 -- 172.21.15.38:0/987434240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796810c170 msgr2=0x7f796819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796810c170 0x7f796819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 -- 172.21.15.38:0/987434240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7968103660 msgr2=0x7f7968198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7968103660 0x7f7968198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 -- 172.21.15.38:0/987434240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796819d380 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796cd10640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f796400c920 tx=0x7f796400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964007c70 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7964007e10 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796819d670 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.675+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964018890 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.676+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79681a4f50 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.677+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968103120 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.678+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7964018a70 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.679+0000 7f795dffb640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 0x7f79500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.679+0000 7f795ffff640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 0x7f79500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.679+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f79640cfe20 con 0x7f7968102d40 2024-09-19T05:48:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.679+0000 7f795ffff640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 0x7f79500a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7958009230 tx=0x7f79580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.684+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7964014030 con 0x7f7968102d40 2024-09-19T05:48:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:35 smithi179 ceph-mon[35781]: pgmap v573: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:35.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:35 smithi047 ceph-mon[35327]: pgmap v573: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:35.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.898+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f796802d7c0 con 0x7f7968102d40 2024-09-19T05:48:35.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.899+0000 7f795dffb640 1 -- 172.21.15.38:0/987434240 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f796409c780 con 0x7f7968102d40 2024-09-19T05:48:35.902 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:48:35.902 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:48:35.902 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:48:35.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 msgr2=0x7f79500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:35.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 0x7f79500a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7958009230 tx=0x7f79580058d0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 msgr2=0x7f79681983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f796400c920 tx=0x7f796400cde0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 shutdown_connections 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f79500a3240 0x7f79500a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796810c170 0x7f796819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7968103660 0x7f7968198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 --2- 172.21.15.38:0/987434240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7968102d40 0x7f79681983d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 >> 172.21.15.38:0/987434240 conn(0x7f79680fc7a0 msgr2=0x7f79680fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 shutdown_connections 2024-09-19T05:48:35.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:35.901+0000 7f796ef74640 1 -- 172.21.15.38:0/987434240 wait complete. 2024-09-19T05:48:36.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:36 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/987434240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:48:36.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:36 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/987434240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:48:36.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:36 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/987434240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:48:36.884 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T05:48:36.884 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T05:48:37.149 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:48:37.561 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:37 smithi038 ceph-mon[31227]: pgmap v574: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 -- 172.21.15.38:0/57609466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 msgr2=0x7fb3301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 --2- 172.21.15.38:0/57609466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3301021c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009900 tx=0x7fb32c02a410 comp rx=0 tx=0).stop 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 -- 172.21.15.38:0/57609466 shutdown_connections 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 --2- 172.21.15.38:0/57609466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb33010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 --2- 172.21.15.38:0/57609466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 0x7fb3301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 --2- 172.21.15.38:0/57609466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3301021c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 -- 172.21.15.38:0/57609466 >> 172.21.15.38:0/57609466 conn(0x7fb3300fd8c0 msgr2=0x7fb3300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 -- 172.21.15.38:0/57609466 shutdown_connections 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 -- 172.21.15.38:0/57609466 wait complete. 2024-09-19T05:48:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.651+0000 7fb337e1b640 1 Processor -- start 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 -- start start 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 0x7fb330074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb330074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb330111f70 con 0x7fb330101de0 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb330111df0 con 0x7fb330106f10 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb337e1b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301120f0 con 0x7fb330102790 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39594/0 (socket says 172.21.15.38:39594) 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 -- 172.21.15.38:0/1159510822 learned_addr learned my addr 172.21.15.38:0/1159510822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:37.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb330074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb3353b6640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 0x7fb330074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 -- 172.21.15.38:0/1159510822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 msgr2=0x7fb330074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 0x7fb330074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 -- 172.21.15.38:0/1159510822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 msgr2=0x7fb330074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb330074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb335bb7640 1 -- 172.21.15.38:0/1159510822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301a68f0 con 0x7fb330101de0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.652+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb330074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb32c006010 tx=0x7fb32c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb326ffd640 1 -- 172.21.15.38:0/1159510822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c03f030 con 0x7fb330101de0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb326ffd640 1 -- 172.21.15.38:0/1159510822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32c003ce0 con 0x7fb330101de0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301a6b80 con 0x7fb330101de0 2024-09-19T05:48:37.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb326ffd640 1 -- 172.21.15.38:0/1159510822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c035070 con 0x7fb330101de0 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.653+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301a7060 con 0x7fb330101de0 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.655+0000 7fb326ffd640 1 -- 172.21.15.38:0/1159510822 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb32c042030 con 0x7fb330101de0 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.655+0000 7fb33a1f4740 1 Processor -- start 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.655+0000 7fb33a1f4740 1 -- start start 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9e170 0x562fb9ba84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9baae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562fb9b02160 con 0x562fb9b9d850 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562fb9aa9720 con 0x562fb9b9e170 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562fb9afe120 con 0x562fb9ba8a30 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39606/0 (socket says 172.21.15.38:39606) 2024-09-19T05:48:37.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 -- 172.21.15.38:0/1563167098 learned_addr learned my addr 172.21.15.38:0/1563167098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9e170 0x562fb9ba84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3353b6640 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9baae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 -- 172.21.15.38:0/1563167098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 msgr2=0x562fb9baae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9baae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 -- 172.21.15.38:0/1563167098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9e170 msgr2=0x562fb9ba84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9e170 0x562fb9ba84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 -- 172.21.15.38:0/1563167098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb9873ba0 con 0x562fb9b9d850 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb326ffd640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 0x7fb31c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 0x7fb31c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.656+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb328009fd0 tx=0x7fb32800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e552470f65db9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb326ffd640 1 -- 172.21.15.38:0/1159510822 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb32c0f0240 con 0x7fb330101de0 2024-09-19T05:48:37.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb31bfff640 1 -- 172.21.15.38:0/1563167098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328015070 con 0x562fb9b9d850 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb31bfff640 1 -- 172.21.15.38:0/1563167098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb328011be0 con 0x562fb9b9d850 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb31bfff640 1 -- 172.21.15.38:0/1563167098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328010c30 con 0x562fb9b9d850 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1563167098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 msgr2=0x562fb9b9dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb328009fd0 tx=0x7fb32800f7f0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb335bb7640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 0x7fb31c0a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb32c042000 tx=0x7fb32c02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1563167098 shutdown_connections 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9baae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9e170 0x562fb9ba84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1563167098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9b9dc30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.657+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1563167098 >> 172.21.15.38:0/1563167098 conn(0x562fb9b99de0 msgr2=0x562fb9b9a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1563167098 shutdown_connections 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1563167098 wait complete. 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- start start 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 Processor -- start 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9ba5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9ba63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562fb9bab600 con 0x562fb9b9d850 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562fb9a8a900 con 0x562fb9ba8a30 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562fb9bab780 con 0x562fb9b9e170 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.658+0000 7fb33a1f4740 10 client.0 _ll_register_callbacks cb 0x562fb9a88780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:48:37.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39612/0 (socket says 172.21.15.38:39612) 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 -- 172.21.15.38:0/848009613 learned_addr learned my addr 172.21.15.38:0/848009613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3353b6640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9ba5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9ba63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 -- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 msgr2=0x562fb9ba5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9ba5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 -- 172.21.15.38:0/848009613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 msgr2=0x562fb9ba63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9ba63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 -- 172.21.15.38:0/848009613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb9acac50 con 0x562fb9b9d850 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3353b6640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9ba5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3363b8640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb328000f80 tx=0x7fb3280108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c318d8d8219e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328015050 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb3280109b0 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.659+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb328011be0 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb328032050 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 10 client.15738 fetch_fsmap learned FSMap version 28 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fb9bb7310 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb328011be0 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb328027020 con 0x562fb9b9d850 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 10 client.15738 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 20 client.15738 populate_metadata read hostname 'smithi038' 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9bb87d0 0x562fb9bbabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 4 client.15738 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:48:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fb9b0f890 con 0x562fb9bb87d0 2024-09-19T05:48:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.660+0000 7fb3353b6640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9bb87d0 0x562fb9bbabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.661+0000 7fb3353b6640 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9bb87d0 0x562fb9bbabc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59545687fade4de server_cookie=9d6e673900be6e81 in_seq=0 out_seq=0 2024-09-19T05:48:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.661+0000 7fb3197fa640 10 client.15738 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb3197fa640 1 -- 172.21.15.38:0/848009613 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fb9b0f890 con 0x562fb9bb87d0 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb3197fa640 10 client.15738 handle_command_reply: tid=0 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 client.15738 shutdown 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 msgr2=0x562fb9ba5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb328000f80 tx=0x7fb3280108b0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 shutdown_connections 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9bb87d0 0x562fb9bbabc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9ba63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9ba5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb3197fa640 0 client.15738 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/848009613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba5940 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 >> 172.21.15.38:0/848009613 conn(0x562fb9b99de0 msgr2=0x562fb9b9fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 shutdown_connections 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 1 -- 172.21.15.38:0/848009613 wait complete. 2024-09-19T05:48:37.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.662+0000 7fb33a1f4740 20 client.15738 trim_cache size 0 max 16384 2024-09-19T05:48:37.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 Processor -- start 2024-09-19T05:48:37.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 -- start start 2024-09-19T05:48:37.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9bc0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9bc06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b0f60 con 0x562fb9b9d850 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b2aa0 con 0x562fb9ba8a30 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b1630 con 0x562fb9b9e170 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb3363b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39620/0 (socket says 172.21.15.38:39620) 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.679+0000 7fb3363b8640 1 -- 172.21.15.38:0/1263868154 learned_addr learned my addr 172.21.15.38:0/1263868154 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9bc06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3353b6640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9bc0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 -- 172.21.15.38:0/1263868154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 msgr2=0x562fb9bc0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9bc0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 -- 172.21.15.38:0/1263868154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 msgr2=0x562fb9bc06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9bc06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 -- 172.21.15.38:0/1263868154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb9b9dfa0 con 0x562fb9b9d850 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3353b6640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9bc0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb3363b8640 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb3280105c0 tx=0x7fb328005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb54f1e11e900b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb31b7fe640 1 -- 172.21.15.38:0/1263868154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32803e070 con 0x562fb9b9d850 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb31b7fe640 1 -- 172.21.15.38:0/1263868154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32801e030 con 0x562fb9b9d850 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.680+0000 7fb31b7fe640 1 -- 172.21.15.38:0/1263868154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328015070 con 0x562fb9b9d850 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1263868154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 msgr2=0x562fb9ba78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb3280105c0 tx=0x7fb328005cf0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1263868154 shutdown_connections 2024-09-19T05:48:37.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9ba8a30 0x562fb9bc06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9b9e170 0x562fb9bc0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/1263868154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9d850 0x562fb9ba78b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1263868154 >> 172.21.15.38:0/1263868154 conn(0x562fb9b99de0 msgr2=0x562fb9ba71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1263868154 shutdown_connections 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- 172.21.15.38:0/1263868154 wait complete. 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 -- start start 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.681+0000 7fb33a1f4740 1 Processor -- start 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9d850 0x562fb9bb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9bad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b23d0 con 0x562fb9b9e170 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301b1d00 con 0x562fb9b9d850 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3301aef40 con 0x562fb9ba8a30 2024-09-19T05:48:37.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 10 client.0 _ll_register_callbacks cb 0x7fb328009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39632/0 (socket says 172.21.15.38:39632) 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 -- 172.21.15.38:0/438089316 learned_addr learned my addr 172.21.15.38:0/438089316 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3363b8640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9d850 0x562fb9bb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3363b8640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9bad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 -- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 msgr2=0x562fb9bad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9bad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 -- 172.21.15.38:0/438089316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9d850 msgr2=0x562fb9bb0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9d850 0x562fb9bb0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 -- 172.21.15.38:0/438089316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb9badb60 con 0x562fb9b9e170 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3353b6640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb330075f80 tx=0x7fb32000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f91b86ef181e82f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3197fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:48:37.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32001d670 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb32001d810 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.682+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32801e030 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb32002d050 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fb9c23d80 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb32801e030 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb320010430 con 0x562fb9b9e170 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 10 client.15744 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 20 client.15744 populate_metadata read hostname 'smithi038' 2024-09-19T05:48:37.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9c24550 0x562fb9c26970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:48:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 4 client.15744 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T05:48:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x562fb9b09be0 con 0x562fb9c24550 2024-09-19T05:48:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.683+0000 7fb3363b8640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9c24550 0x562fb9c26970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:48:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.684+0000 7fb3363b8640 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9c24550 0x562fb9c26970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce73be32e7cc7800 server_cookie=93265e212b447b05 in_seq=0 out_seq=0 2024-09-19T05:48:37.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.684+0000 7fb3197fa640 10 client.15744 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb3197fa640 1 -- 172.21.15.38:0/438089316 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562fb9b09be0 con 0x562fb9c24550 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb3197fa640 10 client.15744 handle_command_reply: tid=0 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 client.15744 shutdown 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 msgr2=0x562fb9bb12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb330075f80 tx=0x7fb32000be80 comp rx=0 tx=0).stop 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 shutdown_connections 2024-09-19T05:48:37.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562fb9c24550 0x562fb9c26970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562fb9ba8a30 0x562fb9bad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562fb9b9e170 0x562fb9bb12e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb3197fa640 0 client.15744 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 --2- 172.21.15.38:0/438089316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fb9b9d850 0x562fb9bb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 >> 172.21.15.38:0/438089316 conn(0x562fb9b99de0 msgr2=0x562fb9b98ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 shutdown_connections 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 1 -- 172.21.15.38:0/438089316 wait complete. 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.694+0000 7fb33a1f4740 20 client.15744 trim_cache size 0 max 16384 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:48:37.698 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:48:37.699 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:48:37.699 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 26, 2024-09-19T05:48:37.699 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:48:37.699 INFO:teuthology.orchestra.run.smithi038.stdout: "a913c5a5": { 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 26, 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 145.8, 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:48:37.700 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:48:37.701 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 msgr2=0x7fb31c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 0x7fb31c0a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb32c042000 tx=0x7fb32c02a410 comp rx=0 tx=0).stop 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 msgr2=0x7fb3300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb32c006010 tx=0x7fb32c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 shutdown_connections 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb31c0a3240 0x7fb31c0a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330106f10 0x7fb330074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb330102790 0x7fb330074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 --2- 172.21.15.38:0/1159510822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb330101de0 0x7fb3300744f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:48:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 >> 172.21.15.38:0/1159510822 conn(0x7fb3300fd8c0 msgr2=0x7fb3300fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:48:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 shutdown_connections 2024-09-19T05:48:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:48:37.695+0000 7fb337e1b640 1 -- 172.21.15.38:0/1159510822 wait complete. 2024-09-19T05:48:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:37 smithi179 ceph-mon[35781]: pgmap v574: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:37.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:37 smithi047 ceph-mon[35327]: pgmap v574: 129 pgs: 129 active+clean; 3.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:48:38.442 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a913c5a5', quiesce took 3.5 seconds. Will release after: 22.25 2024-09-19T05:48:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:39 smithi179 ceph-mon[35781]: pgmap v575: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:39.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:39 smithi047 ceph-mon[35327]: pgmap v575: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:39.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:39 smithi038 ceph-mon[31227]: pgmap v575: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:48:41.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:41 smithi038 ceph-mon[31227]: pgmap v576: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:41 smithi179 ceph-mon[35781]: pgmap v576: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:41.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:41 smithi047 ceph-mon[35327]: pgmap v576: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:48:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:43 smithi179 ceph-mon[35781]: pgmap v577: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:48:43.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:43 smithi047 ceph-mon[35327]: pgmap v577: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:48:43.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:43 smithi038 ceph-mon[31227]: pgmap v577: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:48:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:45 smithi179 ceph-mon[35781]: pgmap v578: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:48:45.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:45 smithi047 ceph-mon[35327]: pgmap v578: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:48:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:45 smithi038 ceph-mon[31227]: pgmap v578: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-09-19T05:48:47.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:47 smithi047 ceph-mon[35327]: pgmap v579: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-19T05:48:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:47 smithi038 ceph-mon[31227]: pgmap v579: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-19T05:48:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:47 smithi179 ceph-mon[35781]: pgmap v579: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-09-19T05:48:49.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:49 smithi047 ceph-mon[35327]: pgmap v580: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-19T05:48:49.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:49 smithi038 ceph-mon[31227]: pgmap v580: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-19T05:48:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:49 smithi179 ceph-mon[35781]: pgmap v580: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-19T05:48:51.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:51 smithi047 ceph-mon[35327]: pgmap v581: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T05:48:51.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:51 smithi038 ceph-mon[31227]: pgmap v581: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T05:48:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:51 smithi179 ceph-mon[35781]: pgmap v581: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-19T05:48:53.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:53 smithi047 ceph-mon[35327]: pgmap v582: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:48:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:53 smithi038 ceph-mon[31227]: pgmap v582: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:48:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:53 smithi179 ceph-mon[35781]: pgmap v582: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:48:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:55 smithi038 ceph-mon[31227]: pgmap v583: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:48:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:55 smithi179 ceph-mon[35781]: pgmap v583: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:48:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:55 smithi047 ceph-mon[35327]: pgmap v583: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:48:57.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:57 smithi179 ceph-mon[35781]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:48:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:57 smithi038 ceph-mon[31227]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:48:57.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:57 smithi047 ceph-mon[35327]: pgmap v584: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:48:59.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:48:59 smithi038 ceph-mon[31227]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:48:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:48:59 smithi179 ceph-mon[35781]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:48:59.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:48:59 smithi047 ceph-mon[35327]: pgmap v585: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:49:00.693 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:49:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 -- 172.21.15.38:0/4090867397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c1055b0 msgr2=0x7fc65c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 --2- 172.21.15.38:0/4090867397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c1055b0 0x7fc65c10d130 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc65402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 -- 172.21.15.38:0/4090867397 shutdown_connections 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 --2- 172.21.15.38:0/4090867397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c1055b0 0x7fc65c10d130 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 --2- 172.21.15.38:0/4090867397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 0x7fc65c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 --2- 172.21.15.38:0/4090867397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c100480 0x7fc65c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 -- 172.21.15.38:0/4090867397 >> 172.21.15.38:0/4090867397 conn(0x7fc65c077210 msgr2=0x7fc65c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 -- 172.21.15.38:0/4090867397 shutdown_connections 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 -- 172.21.15.38:0/4090867397 wait complete. 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.462+0000 7fc663840640 1 Processor -- start 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 -- start start 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 0x7fc65c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 0x7fc65c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c110eb0 con 0x7fc65c100480 2024-09-19T05:49:01.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c110d30 con 0x7fc65c100e30 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc663840640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c111030 con 0x7fc65c1055b0 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42532/0 (socket says 172.21.15.38:42532) 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 -- 172.21.15.38:0/1025760061 learned_addr learned my addr 172.21.15.38:0/1025760061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc660ddb640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 0x7fc65c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc661ddd640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 0x7fc65c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 -- 172.21.15.38:0/1025760061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 msgr2=0x7fc65c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 0x7fc65c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 -- 172.21.15.38:0/1025760061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 msgr2=0x7fc65c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 0x7fc65c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 -- 172.21.15.38:0/1025760061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c1a6930 con 0x7fc65c100480 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc661ddd640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 0x7fc65c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:49:01.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.463+0000 7fc6615dc640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc65800b4a0 tx=0x7fc65800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.464+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658014070 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.464+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc658010af0 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.464+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1a6c20 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.464+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658017d10 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.464+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a7160 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.465+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c100860 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.466+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc658017070 con 0x7fc65c100480 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.466+0000 7fc6527fc640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 0x7fc6480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:01.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.467+0000 7fc660ddb640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 0x7fc6480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.467+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fc6580cf490 con 0x7fc65c100480 2024-09-19T05:49:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.467+0000 7fc660ddb640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 0x7fc6480a5780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005fd0 tx=0x7fc64c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:01.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.470+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc65809bdf0 con 0x7fc65c100480 2024-09-19T05:49:01.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:01 smithi038 ceph-mon[31227]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:01.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.685+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc65c0fdbc0 con 0x7fc65c100480 2024-09-19T05:49:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.686+0000 7fc6527fc640 1 -- 172.21.15.38:0/1025760061 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fc65c0fdbc0 con 0x7fc65c100480 2024-09-19T05:49:01.688 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:49:01.688 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:49:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 msgr2=0x7fc6480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 0x7fc6480a5780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005fd0 tx=0x7fc64c005850 comp rx=0 tx=0).stop 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 msgr2=0x7fc65c0fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc65800b4a0 tx=0x7fc65800b960 comp rx=0 tx=0).stop 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 shutdown_connections 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fc6480a32c0 0x7fc6480a5780 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc65c1055b0 0x7fc65c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc65c100e30 0x7fc65c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 --2- 172.21.15.38:0/1025760061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc65c100480 0x7fc65c0fc690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.688+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 >> 172.21.15.38:0/1025760061 conn(0x7fc65c077210 msgr2=0x7fc65c10dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.689+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 shutdown_connections 2024-09-19T05:49:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:01.689+0000 7fc663840640 1 -- 172.21.15.38:0/1025760061 wait complete. 2024-09-19T05:49:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:01 smithi179 ceph-mon[35781]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:01.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:01 smithi047 ceph-mon[35327]: pgmap v586: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:02.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id a913c5a5 --release --await' 2024-09-19T05:49:02.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id a913c5a5 --release --await 2024-09-19T05:49:02.637 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:02.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:02 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1025760061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:02.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:02 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1025760061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:02.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:02 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1025760061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:03.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 -- 172.21.15.38:0/336379551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90106f10 msgr2=0x7f7e9010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 --2- 172.21.15.38:0/336379551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90106f10 0x7f7e9010ea90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009900 tx=0x7f7e8802a410 comp rx=0 tx=0).stop 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 -- 172.21.15.38:0/336379551 shutdown_connections 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 --2- 172.21.15.38:0/336379551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90106f10 0x7f7e9010ea90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 --2- 172.21.15.38:0/336379551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90102790 0x7f7e901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 --2- 172.21.15.38:0/336379551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90101de0 0x7f7e901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 -- 172.21.15.38:0/336379551 >> 172.21.15.38:0/336379551 conn(0x7f7e900fd8c0 msgr2=0x7f7e900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.137+0000 7f7e9578f640 1 -- 172.21.15.38:0/336379551 shutdown_connections 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 -- 172.21.15.38:0/336379551 wait complete. 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 Processor -- start 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 -- start start 2024-09-19T05:49:03.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90102790 0x7f7e901988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 0x7f7e9019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90111f70 con 0x7f7e90101de0 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90111df0 con 0x7f7e90102790 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.138+0000 7f7e9578f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901120f0 con 0x7f7e90106f10 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42550/0 (socket says 172.21.15.38:42550) 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 -- 172.21.15.38:0/954298147 learned_addr learned my addr 172.21.15.38:0/954298147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90102790 0x7f7e901988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 0x7f7e9019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 -- 172.21.15.38:0/954298147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 msgr2=0x7f7e9019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 0x7f7e9019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 -- 172.21.15.38:0/954298147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90102790 msgr2=0x7f7e901988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90102790 0x7f7e901988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 -- 172.21.15.38:0/954298147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9019d310 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 0x7f7e9019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e8effd640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cce0 tx=0x7f7e8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e87fff640 1 -- 172.21.15.38:0/954298147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000f650 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9019d600 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e87fff640 1 -- 172.21.15.38:0/954298147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8000fcd0 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.139+0000 7f7e87fff640 1 -- 172.21.15.38:0/954298147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80018c70 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.140+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e901a4ee0 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e87fff640 1 -- 172.21.15.38:0/954298147 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7e8000f7f0 con 0x7f7e90101de0 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 Processor -- start 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 -- start start 2024-09-19T05:49:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6980 0x55c32b4e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c32b43b620 con 0x55c32b4d6060 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c32b3e2e50 con 0x55c32b4d6980 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.142+0000 7f7e97b68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c32b4375e0 con 0x55c32b4e1240 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e87fff640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 0x7f7e7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42566/0 (socket says 172.21.15.38:42566) 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/779044447 learned_addr learned my addr 172.21.15.38:0/779044447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e87fff640 1 -- 172.21.15.38:0/954298147 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7e800d01d0 con 0x7f7e90101de0 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8effd640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6980 0x55c32b4e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 0x7f7e7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/779044447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 msgr2=0x55c32b4e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/779044447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6980 msgr2=0x55c32b4e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6980 0x55c32b4e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/779044447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32b3d3d40 con 0x55c32b4d6060 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e88031550 tx=0x7f7e880318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f83d0508fb40c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e857fa640 1 -- 172.21.15.38:0/779044447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88035070 con 0x55c32b4d6060 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e857fa640 1 -- 172.21.15.38:0/779044447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8804c070 con 0x55c32b4d6060 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.143+0000 7f7e857fa640 1 -- 172.21.15.38:0/779044447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88004ae0 con 0x55c32b4d6060 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 -- 172.21.15.38:0/779044447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 msgr2=0x55c32b4d6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 0x7f7e7c0a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e8804b000 tx=0x7f7e88049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e88031550 tx=0x7f7e880318f0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 -- 172.21.15.38:0/779044447 shutdown_connections 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6980 0x55c32b4e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 --2- 172.21.15.38:0/779044447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6060 0x55c32b4d6440 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 -- 172.21.15.38:0/779044447 >> 172.21.15.38:0/779044447 conn(0x55c32b4d25f0 msgr2=0x55c32b4d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 -- 172.21.15.38:0/779044447 shutdown_connections 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.144+0000 7f7e97b68740 1 -- 172.21.15.38:0/779044447 wait complete. 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 -- start start 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 Processor -- start 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 0x55c32b4de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 0x55c32b4deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55c32b4e3e10 con 0x55c32b4d6980 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c32b3c3f10 con 0x55c32b4e1240 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c32b4e3f90 con 0x55c32b4d6060 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e97b68740 10 client.0 _ll_register_callbacks cb 0x55c32b3e7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42576/0 (socket says 172.21.15.38:42576) 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/3391389051 learned_addr learned my addr 172.21.15.38:0/3391389051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 0x55c32b4deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 0x55c32b4de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 msgr2=0x55c32b4de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 0x55c32b4de120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/3391389051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 msgr2=0x55c32b4deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 0x55c32b4deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/3391389051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32b14a580 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 0x55c32b4deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.145+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 0x55c32b4de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800b970 tx=0x7f7e7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99729b9100be93db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78003ea0 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7e78005220 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8804c070 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e78023050 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e97b68740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c32b214960 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8804c070 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.146+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e780125e0 con 0x55c32b4d6980 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:03.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 10 client.15762 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 20 client.15762 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b4f0e20 0x55c32b4f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 4 client.15762 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c32b448b90 con 0x55c32b4f0e20 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b4f0e20 0x55c32b4f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b4f0e20 0x55c32b4f3260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3e2cd1eee9c18f server_cookie=2e0dc11ad9cb5558 in_seq=0 out_seq=0 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.147+0000 7f7e767fc640 10 client.15762 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e767fc640 1 -- 172.21.15.38:0/3391389051 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c32b448b90 con 0x55c32b4f0e20 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e767fc640 10 client.15762 handle_command_reply: tid=0 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 client.15762 shutdown 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 msgr2=0x55c32b4de660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800b970 tx=0x7f7e7800be30 comp rx=0 tx=0).stop 2024-09-19T05:49:03.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 shutdown_connections 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b4f0e20 0x55c32b4f3260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4e1240 0x55c32b4deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4de660 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3391389051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6060 0x55c32b4de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e767fc640 0 client.15762 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 >> 172.21.15.38:0/3391389051 conn(0x55c32b4d25f0 msgr2=0x55c32b4d8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.148+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 shutdown_connections 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.149+0000 7f7e97b68740 1 -- 172.21.15.38:0/3391389051 wait complete. 2024-09-19T05:49:03.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.149+0000 7f7e97b68740 20 client.15762 trim_cache size 0 max 16384 2024-09-19T05:49:03.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 Processor -- start 2024-09-19T05:49:03.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 -- start start 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4d8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901aed70 con 0x55c32b4d6980 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901b08b0 con 0x55c32b4d6060 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e97b68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901af440 con 0x55c32b4e1240 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42588/0 (socket says 172.21.15.38:42588) 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/553845545 learned_addr learned my addr 172.21.15.38:0/553845545 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8effd640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4d8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/553845545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 msgr2=0x55c32b4d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/553845545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 msgr2=0x55c32b4d8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4d8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8e7fc640 1 -- 172.21.15.38:0/553845545 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c32b4e0060 con 0x55c32b4d6980 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.166+0000 7f7e8effd640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4d8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.167+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7e78012980 tx=0x7f7e78003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1a4aad31d32fde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.167+0000 7f7e84ff9640 1 -- 172.21.15.38:0/553845545 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780039c0 con 0x55c32b4d6980 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.167+0000 7f7e84ff9640 1 -- 172.21.15.38:0/553845545 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78016070 con 0x55c32b4d6980 2024-09-19T05:49:03.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.167+0000 7f7e84ff9640 1 -- 172.21.15.38:0/553845545 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78004540 con 0x55c32b4d6980 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- 172.21.15.38:0/553845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 msgr2=0x55c32b4d2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7e78012980 tx=0x7f7e78003ea0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- 172.21.15.38:0/553845545 shutdown_connections 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4e1240 0x55c32b4d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4d6980 0x55c32b4d2c00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- 172.21.15.38:0/553845545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4d8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- 172.21.15.38:0/553845545 >> 172.21.15.38:0/553845545 conn(0x55c32b4d25f0 msgr2=0x55c32b4d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- 172.21.15.38:0/553845545 shutdown_connections 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- 172.21.15.38:0/553845545 wait complete. 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- start start 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 Processor -- start 2024-09-19T05:49:03.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6980 0x55c32b4e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901b01e0 con 0x55c32b4e1240 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901afb10 con 0x55c32b4d6060 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.168+0000 7f7e97b68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901acd50 con 0x55c32b4d6980 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e97b68740 10 client.0 _ll_register_callbacks cb 0x7f7e78004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42598/0 (socket says 172.21.15.38:42598) 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/3694538291 learned_addr learned my addr 172.21.15.38:0/3694538291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8e7fc640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6980 0x55c32b4e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6980 msgr2=0x55c32b4e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6980 0x55c32b4e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/3694538291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 msgr2=0x55c32b4e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 -- 172.21.15.38:0/3694538291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c32b55ada0 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e8f7fe640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7e88031550 tx=0x7f7e88005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47873a69dffc2ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e767fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88060060 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7e880035f0 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.169+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e78016070 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e88067050 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c32b55c0c0 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78016070 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e88004d80 con 0x55c32b4e1240 2024-09-19T05:49:03.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 10 client.15768 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 20 client.15768 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b55c400 0x55c32b55e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 4 client.15768 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a913c5a5", "release": true, "await": true}] 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a913c5a5", "release": true, "await": true}) -- 0x55c32b4430a0 con 0x55c32b55c400 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.170+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b55c400 0x55c32b55e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.171+0000 7f7e8effd640 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b55c400 0x55c32b55e7f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7d57debe32e56a server_cookie=f537e7647d951e38 in_seq=0 out_seq=0 2024-09-19T05:49:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.171+0000 7f7e767fc640 10 client.15768 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e767fc640 1 -- 172.21.15.38:0/3694538291 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c32b4430a0 con 0x55c32b55c400 2024-09-19T05:49:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e767fc640 10 client.15768 handle_command_reply: tid=0 2024-09-19T05:49:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 client.15768 shutdown 2024-09-19T05:49:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 msgr2=0x55c32b4e5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7e88031550 tx=0x7f7e88005e40 comp rx=0 tx=0).stop 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 shutdown_connections 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55c32b55c400 0x55c32b55e7f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c32b4e1240 0x55c32b4e5ca0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c32b4d6980 0x55c32b4e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 --2- 172.21.15.38:0/3694538291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c32b4d6060 0x55c32b4e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 >> 172.21.15.38:0/3694538291 conn(0x55c32b4d25f0 msgr2=0x55c32b4d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e767fc640 0 client.15768 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 shutdown_connections 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 1 -- 172.21.15.38:0/3694538291 wait complete. 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.174+0000 7f7e97b68740 20 client.15768 trim_cache size 0 max 16384 2024-09-19T05:49:03.178 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 28, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "a913c5a5": { 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 28, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:49:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 145.8, 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:03.180 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 msgr2=0x7f7e7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 0x7f7e7c0a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e8804b000 tx=0x7f7e88049000 comp rx=0 tx=0).stop 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 msgr2=0x7f7e90198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cce0 tx=0x7f7e8000a620 comp rx=0 tx=0).stop 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 shutdown_connections 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7e7c0a3240 0x7f7e7c0a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e90106f10 0x7f7e9019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90102790 0x7f7e901988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 --2- 172.21.15.38:0/954298147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e90101de0 0x7f7e90198360 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 >> 172.21.15.38:0/954298147 conn(0x7f7e900fd8c0 msgr2=0x7f7e900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 shutdown_connections 2024-09-19T05:49:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:03.175+0000 7f7e9578f640 1 -- 172.21.15.38:0/954298147 wait complete. 2024-09-19T05:49:03.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:03 smithi038 ceph-mon[31227]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T05:49:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:03 smithi179 ceph-mon[35781]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T05:49:03.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a913c5a5', total seconds elapsed: 29.0 2024-09-19T05:49:03.951 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T05:49:03.951 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:49:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:03 smithi047 ceph-mon[35327]: pgmap v587: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-19T05:49:04.210 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 -- 172.21.15.38:0/439046956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 msgr2=0x7ff63010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 --2- 172.21.15.38:0/439046956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63010df40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff62002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 -- 172.21.15.38:0/439046956 shutdown_connections 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 --2- 172.21.15.38:0/439046956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff63010e480 0x7ff630110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 --2- 172.21.15.38:0/439046956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63010df40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 --2- 172.21.15.38:0/439046956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6301008d0 0x7ff630100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 -- 172.21.15.38:0/439046956 >> 172.21.15.38:0/439046956 conn(0x7ff6300fc7a0 msgr2=0x7ff6300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 -- 172.21.15.38:0/439046956 shutdown_connections 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.732+0000 7ff6379c2640 1 -- 172.21.15.38:0/439046956 wait complete. 2024-09-19T05:49:04.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.733+0000 7ff6379c2640 1 Processor -- start 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.733+0000 7ff6379c2640 1 -- start start 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6301008d0 0x7ff63019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 0x7ff63019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff630114190 con 0x7ff630101280 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff630114010 con 0x7ff63010e480 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff6379c2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff630114310 con 0x7ff6301008d0 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42614/0 (socket says 172.21.15.38:42614) 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 -- 172.21.15.38:0/4013865921 learned_addr learned my addr 172.21.15.38:0/4013865921 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff63575e640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6301008d0 0x7ff63019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:04.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff635f5f640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 0x7ff63019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 -- 172.21.15.38:0/4013865921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6301008d0 msgr2=0x7ff63019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6301008d0 0x7ff63019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 -- 172.21.15.38:0/4013865921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 msgr2=0x7ff63019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 0x7ff63019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 -- 172.21.15.38:0/4013865921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63019f540 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff635f5f640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 0x7ff63019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.734+0000 7ff634f5d640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff620005ab0 tx=0x7ff6200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.735+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62003f030 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.735+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63019f7d0 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.735+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff620003ce0 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.735+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620035070 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.735+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301a7110 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.736+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63006b070 con 0x7ff630101280 2024-09-19T05:49:04.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.737+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff620042030 con 0x7ff630101280 2024-09-19T05:49:04.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.737+0000 7ff6267fc640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 0x7ff61c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:04.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.738+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7ff6200f0250 con 0x7ff630101280 2024-09-19T05:49:04.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.738+0000 7ff63575e640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 0x7ff61c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:04.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.738+0000 7ff63575e640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 0x7ff61c0a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00abe0 tx=0x7ff62c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:04.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.742+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff6200bcbb0 con 0x7ff630101280 2024-09-19T05:49:04.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.956+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6301a72e0 con 0x7ff630101280 2024-09-19T05:49:04.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.957+0000 7ff6267fc640 1 -- 172.21.15.38:0/4013865921 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7ff6200bc4b0 con 0x7ff630101280 2024-09-19T05:49:04.959 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:49:04.959 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:49:04.959 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:49:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 msgr2=0x7ff61c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:04.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 0x7ff61c0a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00abe0 tx=0x7ff62c00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 msgr2=0x7ff63019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff620005ab0 tx=0x7ff6200042f0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 shutdown_connections 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff61c0a3240 0x7ff61c0a5700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63010e480 0x7ff63019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff630101280 0x7ff63019aad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 --2- 172.21.15.38:0/4013865921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6301008d0 0x7ff63019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 >> 172.21.15.38:0/4013865921 conn(0x7ff6300fc7a0 msgr2=0x7ff6300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 shutdown_connections 2024-09-19T05:49:04.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:04.960+0000 7ff6379c2640 1 -- 172.21.15.38:0/4013865921 wait complete. 2024-09-19T05:49:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:05 smithi038 ceph-mon[31227]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:05 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4013865921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:05.683 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T05:49:05.683 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T05:49:05.946 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:05 smithi179 ceph-mon[35781]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:05 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/4013865921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:05 smithi047 ceph-mon[35327]: pgmap v588: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:05.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:05 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/4013865921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:06.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 -- 172.21.15.38:0/3338263983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154101df0 msgr2=0x7fe1541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 --2- 172.21.15.38:0/3338263983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154101df0 0x7fe1541021d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe144009ca0 tx=0x7fe14402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 -- 172.21.15.38:0/3338263983 shutdown_connections 2024-09-19T05:49:06.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 --2- 172.21.15.38:0/3338263983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe154106f20 0x7fe15410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 --2- 172.21.15.38:0/3338263983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1541027a0 0x7fe1541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 --2- 172.21.15.38:0/3338263983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154101df0 0x7fe1541021d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 -- 172.21.15.38:0/3338263983 >> 172.21.15.38:0/3338263983 conn(0x7fe1540fd8f0 msgr2=0x7fe1540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 -- 172.21.15.38:0/3338263983 shutdown_connections 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.478+0000 7fe15c604640 1 -- 172.21.15.38:0/3338263983 wait complete. 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 Processor -- start 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 -- start start 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe154101df0 0x7fe1541984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1541027a0 0x7fe154198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111fa0 con 0x7fe154106f20 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111e20 con 0x7fe1541027a0 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.479+0000 7fe15c604640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe154112120 con 0x7fe154101df0 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42624/0 (socket says 172.21.15.38:42624) 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 -- 172.21.15.38:0/809929748 learned_addr learned my addr 172.21.15.38:0/809929748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe154101df0 0x7fe1541984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe159b9f640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1541027a0 0x7fe154198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 -- 172.21.15.38:0/809929748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe154101df0 msgr2=0x7fe1541984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe154101df0 0x7fe1541984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 -- 172.21.15.38:0/809929748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1541027a0 msgr2=0x7fe154198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1541027a0 0x7fe154198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 -- 172.21.15.38:0/809929748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15419d470 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15aba1640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00d750 tx=0x7fe14c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe1537fe640 1 -- 172.21.15.38:0/809929748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c017070 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe1537fe640 1 -- 172.21.15.38:0/809929748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c0097c0 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15419d760 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe1537fe640 1 -- 172.21.15.38:0/809929748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c004d00 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.480+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a5040 con 0x7fe154106f20 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.483+0000 7fe15e9dd740 1 Processor -- start 2024-09-19T05:49:06.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe1537fe640 1 -- 172.21.15.38:0/809929748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe14c009970 con 0x7fe154106f20 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 -- start start 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 0x556b86ed8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86ee34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b86e3d160 con 0x556b86ee3a30 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556b86de4720 con 0x556b86ed8850 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556b86e39120 con 0x556b86ed9170 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe15a3a0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86ee34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42638/0 (socket says 172.21.15.38:42638) 2024-09-19T05:49:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe159b9f640 1 -- 172.21.15.38:0/1370857117 learned_addr learned my addr 172.21.15.38:0/1370857117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.484+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 0x556b86ed8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe1537fe640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 0x7fe13c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 -- 172.21.15.38:0/1370857117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 msgr2=0x556b86ed8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 0x556b86ed8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 -- 172.21.15.38:0/1370857117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 msgr2=0x556b86ee5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 -- 172.21.15.38:0/1370857117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b86baeba0 con 0x556b86ed9170 2024-09-19T05:49:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 0x556b86ed8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15aba1640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 0x7fe13c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86ee34f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe144000f80 tx=0x7fe1440098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95efd8b8933d742c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe1537fe640 1 -- 172.21.15.38:0/809929748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe14c0cf640 con 0x7fe154106f20 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe150ff9640 1 -- 172.21.15.38:0/1370857117 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144049070 con 0x556b86ed9170 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe150ff9640 1 -- 172.21.15.38:0/1370857117 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe144036040 con 0x556b86ed9170 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.485+0000 7fe150ff9640 1 -- 172.21.15.38:0/1370857117 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14402fb20 con 0x556b86ed9170 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1370857117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 msgr2=0x556b86ee34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86ee34f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe144000f80 tx=0x7fe1440098b0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15aba1640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 0x7fe13c0a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe14c09b830 tx=0x7fe14c0962d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1370857117 shutdown_connections 2024-09-19T05:49:06.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86ee34f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1370857117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed8850 0x556b86ed8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1370857117 >> 172.21.15.38:0/1370857117 conn(0x556b86ed4de0 msgr2=0x556b86ed51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1370857117 shutdown_connections 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.486+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1370857117 wait complete. 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 -- start start 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 Processor -- start 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed8850 0x556b86ee0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 0x556b86ee0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b86ee6480 con 0x556b86ee3a30 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556b86dc5900 con 0x556b86ed9170 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556b86ee6600 con 0x556b86ed8850 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15e9dd740 10 client.0 _ll_register_callbacks cb 0x556b86dc3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42650/0 (socket says 172.21.15.38:42650) 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 -- 172.21.15.38:0/38839694 learned_addr learned my addr 172.21.15.38:0/38839694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 0x556b86ee0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed8850 0x556b86ee0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 -- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed8850 msgr2=0x556b86ee0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed8850 0x556b86ee0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 -- 172.21.15.38:0/38839694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 msgr2=0x556b86ee0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 0x556b86ee0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe159b9f640 1 -- 172.21.15.38:0/38839694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b86e05c50 con 0x556b86ee3a30 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.487+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 0x556b86ee0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe159b9f640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe14801aa50 tx=0x7fe1480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a1c3ab5be61ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14801a110 con 0x556b86ee3a30 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe144036040 con 0x556b86ee3a30 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe148013bc0 con 0x556b86ee3a30 2024-09-19T05:49:06.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe148029050 con 0x556b86ee3a30 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe15e9dd740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b86ef65f0 con 0x556b86ee3a30 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.488+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe144036040 con 0x556b86ee3a30 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe148017440 con 0x556b86ee3a30 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 10 client.15786 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 20 client.15786 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86ef7ab0 0x556b86ef9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 4 client.15786 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b86e4a890 con 0x556b86ef7ab0 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86ef7ab0 0x556b86ef9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe139ffb640 10 client.15786 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.489+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86ef7ab0 0x556b86ef9ee0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3981b5bfffb6620 server_cookie=f2bf3bb386d7e785 in_seq=0 out_seq=0 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe139ffb640 1 -- 172.21.15.38:0/38839694 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b86e4a890 con 0x556b86ef7ab0 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe139ffb640 10 client.15786 handle_command_reply: tid=0 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 client.15786 shutdown 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 msgr2=0x556b86ee1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe14801aa50 tx=0x7fe1480046c0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 shutdown_connections 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86ef7ab0 0x556b86ef9ee0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ee3a30 0x556b86ee1390 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ed9170 0x556b86ee0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe139ffb640 0 client.15786 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:06.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/38839694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed8850 0x556b86ee0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 >> 172.21.15.38:0/38839694 conn(0x556b86ed4de0 msgr2=0x556b86edaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 shutdown_connections 2024-09-19T05:49:06.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.490+0000 7fe15e9dd740 1 -- 172.21.15.38:0/38839694 wait complete. 2024-09-19T05:49:06.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.491+0000 7fe15e9dd740 20 client.15786 trim_cache size 0 max 16384 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 Processor -- start 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 -- start start 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86effcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86eddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541acac0 con 0x556b86ed8850 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ae600 con 0x556b86ee3a30 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ad190 con 0x556b86ed9170 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.508+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42656/0 (socket says 172.21.15.38:42656) 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 -- 172.21.15.38:0/1547905975 learned_addr learned my addr 172.21.15.38:0/1547905975 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86effcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86eddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 -- 172.21.15.38:0/1547905975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 msgr2=0x556b86effcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86effcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 -- 172.21.15.38:0/1547905975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 msgr2=0x556b86eddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86eddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 -- 172.21.15.38:0/1547905975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b86ed8680 con 0x556b86ed8850 2024-09-19T05:49:06.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe148009a90 tx=0x7fe148009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67568c5b28d8d58c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe13bfff640 1 -- 172.21.15.38:0/1547905975 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148017440 con 0x556b86ed8850 2024-09-19T05:49:06.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe13bfff640 1 -- 172.21.15.38:0/1547905975 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe148020070 con 0x556b86ed8850 2024-09-19T05:49:06.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.509+0000 7fe13bfff640 1 -- 172.21.15.38:0/1547905975 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148013e20 con 0x556b86ed8850 2024-09-19T05:49:06.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.510+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1547905975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 msgr2=0x556b86eff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.510+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe148009a90 tx=0x7fe148009a20 comp rx=0 tx=0).stop 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.510+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1547905975 shutdown_connections 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.510+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86eddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.510+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86effcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1547905975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eff7b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1547905975 >> 172.21.15.38:0/1547905975 conn(0x556b86ed4de0 msgr2=0x556b86ee2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1547905975 shutdown_connections 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1547905975 wait complete. 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 -- start start 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.511+0000 7fe15e9dd740 1 Processor -- start 2024-09-19T05:49:06.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86eec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86f61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541adf30 con 0x556b86ed8850 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ad860 con 0x556b86ee3a30 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541aaaa0 con 0x556b86ed9170 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15e9dd740 10 client.0 _ll_register_callbacks cb 0x7fe148009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42666/0 (socket says 172.21.15.38:42666) 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 -- 172.21.15.38:0/1504690432 learned_addr learned my addr 172.21.15.38:0/1504690432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86eec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86f61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 -- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 msgr2=0x556b86eec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86eec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 -- 172.21.15.38:0/1504690432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 msgr2=0x556b86f61780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86f61780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 -- 172.21.15.38:0/1504690432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b86f61cc0 con 0x556b86ed8850 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe159b9f640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe1480099f0 tx=0x7fe148009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6bf6fa549b09a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.512+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148032070 con 0x556b86ed8850 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe148013bc0 con 0x556b86ed8850 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe148020070 con 0x556b86ed8850 2024-09-19T05:49:06.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe14801a290 con 0x556b86ed8850 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 10 client.15792 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b86f62ea0 con 0x556b86ed8850 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe148020070 con 0x556b86ed8850 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe148018c60 con 0x556b86ed8850 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 10 client.15792 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 20 client.15792 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86f63520 0x556b86f65910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 4 client.15792 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x556b86e44be0 con 0x556b86f63520 2024-09-19T05:49:06.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.513+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86f63520 0x556b86f65910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.514+0000 7fe15a3a0640 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86f63520 0x556b86f65910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0d6e70f9698702 server_cookie=8c8a73e179543f63 in_seq=0 out_seq=0 2024-09-19T05:49:06.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.514+0000 7fe139ffb640 10 client.15792 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe139ffb640 1 -- 172.21.15.38:0/1504690432 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b86e44be0 con 0x556b86f63520 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe139ffb640 10 client.15792 handle_command_reply: tid=0 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 client.15792 shutdown 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 msgr2=0x556b86eec070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe1480099f0 tx=0x7fe148009ac0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 shutdown_connections 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556b86f63520 0x556b86f65910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556b86ee3a30 0x556b86f61780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b86ed9170 0x556b86eec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 --2- 172.21.15.38:0/1504690432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b86ed8850 0x556b86eec070 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 >> 172.21.15.38:0/1504690432 conn(0x556b86ed4de0 msgr2=0x556b86ee2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe139ffb640 0 client.15792 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 shutdown_connections 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 1 -- 172.21.15.38:0/1504690432 wait complete. 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.524+0000 7fe15e9dd740 20 client.15792 trim_cache size 0 max 16384 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:49:06.527 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 30, 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "814baed3": { 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 30, 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:06.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 145.8, 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:06.529 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:06.530 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:06.530 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:06.530 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:06.530 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:06.530 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:49:06.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 msgr2=0x7fe13c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 0x7fe13c0a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe14c09b830 tx=0x7fe14c0962d0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 msgr2=0x7fe15419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00d750 tx=0x7fe14c00db50 comp rx=0 tx=0).stop 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 shutdown_connections 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe13c0a3240 0x7fe13c0a5700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe154106f20 0x7fe15419cd90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1541027a0 0x7fe154198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 --2- 172.21.15.38:0/809929748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe154101df0 0x7fe1541984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 >> 172.21.15.38:0/809929748 conn(0x7fe1540fd8f0 msgr2=0x7fe1540fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 shutdown_connections 2024-09-19T05:49:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:06.525+0000 7fe15c604640 1 -- 172.21.15.38:0/809929748 wait complete. 2024-09-19T05:49:07.430 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '814baed3', quiesce took 3.5 seconds. Will release after: 22.25 2024-09-19T05:49:07.680 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:07 smithi038 ceph-mon[31227]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:07 smithi179 ceph-mon[35781]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:07.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:07 smithi047 ceph-mon[35327]: pgmap v589: 129 pgs: 129 active+clean; 3.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: pgmap v590: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:49:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:49:09.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:49:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: pgmap v590: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:49:09.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: pgmap v590: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:49:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:49:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:49:09.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:49:10.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:10 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi179 to 176.0M 2024-09-19T05:49:10.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:10 smithi038 ceph-mon[31227]: Unable to set osd_memory_target on smithi179 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T05:49:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:10 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi179 to 176.0M 2024-09-19T05:49:10.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:10 smithi179 ceph-mon[35781]: Unable to set osd_memory_target on smithi179 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T05:49:10.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:10 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi179 to 176.0M 2024-09-19T05:49:10.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:10 smithi047 ceph-mon[35327]: Unable to set osd_memory_target on smithi179 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-19T05:49:11.795 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:11 smithi038 ceph-mon[31227]: pgmap v591: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-19T05:49:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:11 smithi179 ceph-mon[35781]: pgmap v591: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-19T05:49:11.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:11 smithi047 ceph-mon[35327]: pgmap v591: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-19T05:49:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:13 smithi038 ceph-mon[31227]: pgmap v592: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-19T05:49:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:13 smithi179 ceph-mon[35781]: pgmap v592: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-19T05:49:13.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:13 smithi047 ceph-mon[35327]: pgmap v592: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-19T05:49:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:15 smithi038 ceph-mon[31227]: pgmap v593: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:49:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:15 smithi179 ceph-mon[35781]: pgmap v593: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:49:15.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:15 smithi047 ceph-mon[35327]: pgmap v593: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:49:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:17 smithi038 ceph-mon[31227]: pgmap v594: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T05:49:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:17 smithi179 ceph-mon[35781]: pgmap v594: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T05:49:17.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:17 smithi047 ceph-mon[35327]: pgmap v594: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-19T05:49:19.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:19 smithi038 ceph-mon[31227]: pgmap v595: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:49:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:19 smithi179 ceph-mon[35781]: pgmap v595: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:49:19.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:19 smithi047 ceph-mon[35327]: pgmap v595: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:49:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:21 smithi038 ceph-mon[31227]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:49:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:21 smithi179 ceph-mon[35781]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:49:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:21 smithi047 ceph-mon[35327]: pgmap v596: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:49:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:23 smithi038 ceph-mon[31227]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:49:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:23 smithi179 ceph-mon[35781]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:49:23.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:23 smithi047 ceph-mon[35327]: pgmap v597: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:49:25.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:25 smithi038 ceph-mon[31227]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:49:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:25 smithi179 ceph-mon[35781]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:49:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:25 smithi047 ceph-mon[35327]: pgmap v598: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T05:49:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:27 smithi038 ceph-mon[31227]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:49:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:27 smithi179 ceph-mon[35781]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:49:27.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:27 smithi047 ceph-mon[35327]: pgmap v599: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T05:49:29.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:49:29.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:29 smithi038 ceph-mon[31227]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:49:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:29 smithi179 ceph-mon[35781]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:49:29.955 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:29.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:29 smithi047 ceph-mon[35327]: pgmap v600: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:49:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 -- 172.21.15.38:0/2686337466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102570 msgr2=0x7f9b001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2686337466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102570 0x7f9b001029b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009900 tx=0x7f9af402a480 comp rx=0 tx=0).stop 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 -- 172.21.15.38:0/2686337466 shutdown_connections 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2686337466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102ef0 0x7f9b0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2686337466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102570 0x7f9b001029b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2686337466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 -- 172.21.15.38:0/2686337466 >> 172.21.15.38:0/2686337466 conn(0x7f9b000fc7d0 msgr2=0x7f9b000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 -- 172.21.15.38:0/2686337466 shutdown_connections 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.444+0000 7f9b0779b640 1 -- 172.21.15.38:0/2686337466 wait complete. 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 Processor -- start 2024-09-19T05:49:30.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 -- start start 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102570 0x7f9b000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00110f50 con 0x7f9b00102ef0 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00110dd0 con 0x7f9b00102570 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b0779b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001110d0 con 0x7f9b00100e20 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33704/0 (socket says 172.21.15.38:33704) 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 -- 172.21.15.38:0/2694454505 learned_addr learned my addr 172.21.15.38:0/2694454505 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b04d36640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102570 0x7f9b000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05537640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 -- 172.21.15.38:0/2694454505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 msgr2=0x7f9b000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 -- 172.21.15.38:0/2694454505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102570 msgr2=0x7f9b000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102570 0x7f9b000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05d38640 1 -- 172.21.15.38:0/2694454505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b001a6a60 con 0x7f9b00102ef0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.445+0000 7f9b05537640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9b05d38640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9af800eaa0 tx=0x7f9af800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af800cdb0 con 0x7f9b00102ef0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8005020 con 0x7f9b00102ef0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af80116b0 con 0x7f9b00102ef0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b001a6d50 con 0x7f9b00102ef0 2024-09-19T05:49:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.446+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001a7210 con 0x7f9b00102ef0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.447+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b00101200 con 0x7f9b00102ef0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.448+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9af8005560 con 0x7f9b00102ef0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.449+0000 7f9afe7fc640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 0x7f9ae80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.449+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f9af80cfee0 con 0x7f9b00102ef0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.449+0000 7f9b05537640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 0x7f9ae80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.449+0000 7f9b05537640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 0x7f9ae80a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005d00 tx=0x7f9af0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.453+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9af8016030 con 0x7f9b00102ef0 2024-09-19T05:49:30.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.672+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b0010a0a0 con 0x7f9b00102ef0 2024-09-19T05:49:30.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.672+0000 7f9afe7fc640 1 -- 172.21.15.38:0/2694454505 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f9af809c840 con 0x7f9b00102ef0 2024-09-19T05:49:30.675 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:49:30.676 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:49:30.676 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 msgr2=0x7f9ae80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 0x7f9ae80a5700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005d00 tx=0x7f9af0009550 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 msgr2=0x7f9b000fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9af800eaa0 tx=0x7f9af800ef60 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 shutdown_connections 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f9ae80a3240 0x7f9ae80a5700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b00102ef0 0x7f9b000fd700 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b00102570 0x7f9b000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 --2- 172.21.15.38:0/2694454505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b00100e20 0x7f9b000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 >> 172.21.15.38:0/2694454505 conn(0x7f9b000fc7d0 msgr2=0x7f9b0006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 shutdown_connections 2024-09-19T05:49:30.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:30.675+0000 7f9b0779b640 1 -- 172.21.15.38:0/2694454505 wait complete. 2024-09-19T05:49:31.361 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 814baed3 --release --await' 2024-09-19T05:49:31.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 814baed3 --release --await 2024-09-19T05:49:31.625 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:49:31.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:31 smithi038 ceph-mon[31227]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:49:31.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:31 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2694454505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:31 smithi179 ceph-mon[35781]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:49:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:31 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2694454505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:31.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:31 smithi047 ceph-mon[35327]: pgmap v601: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:49:31.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:31 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2694454505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- 172.21.15.38:0/2079703333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0102ef0 msgr2=0x7fb9a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 --2- 172.21.15.38:0/2079703333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0102ef0 0x7fb9a010da40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb998009ca0 tx=0x7fb99802a950 comp rx=0 tx=0).stop 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- 172.21.15.38:0/2079703333 shutdown_connections 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 --2- 172.21.15.38:0/2079703333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0102ef0 0x7fb9a010da40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 --2- 172.21.15.38:0/2079703333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 0x7fb9a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 --2- 172.21.15.38:0/2079703333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0100e20 0x7fb9a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- 172.21.15.38:0/2079703333 >> 172.21.15.38:0/2079703333 conn(0x7fb9a00fc7d0 msgr2=0x7fb9a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- 172.21.15.38:0/2079703333 shutdown_connections 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- 172.21.15.38:0/2079703333 wait complete. 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 Processor -- start 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.129+0000 7fb9a782c640 1 -- start start 2024-09-19T05:49:32.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 0x7fb9a0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0102ef0 0x7fb9a01a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0110fa0 con 0x7fb9a0100e20 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0110e20 con 0x7fb9a0102ef0 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a782c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0111120 con 0x7fb9a0102570 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33726/0 (socket says 172.21.15.38:33726) 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 -- 172.21.15.38:0/956695928 learned_addr learned my addr 172.21.15.38:0/956695928 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 0x7fb9a0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0102ef0 0x7fb9a01a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 -- 172.21.15.38:0/956695928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 msgr2=0x7fb9a0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 0x7fb9a0199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 -- 172.21.15.38:0/956695928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0102ef0 msgr2=0x7fb9a01a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0102ef0 0x7fb9a01a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 -- 172.21.15.38:0/956695928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01a48f0 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c890 tx=0x7fb99c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.130+0000 7fb9967fc640 1 -- 172.21.15.38:0/956695928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004220 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.131+0000 7fb9967fc640 1 -- 172.21.15.38:0/956695928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c0043c0 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.131+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a4ac0 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.131+0000 7fb9967fc640 1 -- 172.21.15.38:0/956695928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004a00 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.131+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a4fd0 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.133+0000 7fb9967fc640 1 -- 172.21.15.38:0/956695928 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb99c0079b0 con 0x7fb9a0100e20 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.133+0000 7fb9a9c05740 1 Processor -- start 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 -- start start 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45842d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45845630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e4579d620 con 0x562e45838980 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562e45744e50 con 0x562e45838060 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e457995e0 con 0x562e45843240 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a55c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45842d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:45896/0 (socket says 172.21.15.38:45896) 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/2373010341 learned_addr learned my addr 172.21.15.38:0/2373010341 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45845630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9967fc640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/2373010341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 msgr2=0x562e45845630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45845630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/2373010341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 msgr2=0x562e45842d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45842d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.134+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/2373010341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e45735d40 con 0x562e45838060 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb9967fc640 1 -- 172.21.15.38:0/956695928 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb99c0cf950 con 0x7fb9a0100e20 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb998035000 tx=0x7fb99802fff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475cb5089170fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb98b7fe640 1 -- 172.21.15.38:0/2373010341 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998034040 con 0x562e45838060 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 0x7fb98c0a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb99804d000 tx=0x7fb99804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb98b7fe640 1 -- 172.21.15.38:0/2373010341 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998004520 con 0x562e45838060 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.135+0000 7fb98b7fe640 1 -- 172.21.15.38:0/2373010341 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980469a0 con 0x562e45838060 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2373010341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 msgr2=0x562e45838440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb998035000 tx=0x7fb99802fff0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2373010341 shutdown_connections 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45845630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45842d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2373010341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45838440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2373010341 >> 172.21.15.38:0/2373010341 conn(0x562e458345f0 msgr2=0x562e458349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2373010341 shutdown_connections 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2373010341 wait complete. 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 -- start start 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.136+0000 7fb9a9c05740 1 Processor -- start 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45840120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45840ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562e45845e10 con 0x562e45838980 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562e45725f10 con 0x562e45838060 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562e45845f90 con 0x562e45843240 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a9c05740 10 client.0 _ll_register_callbacks cb 0x562e45749db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:32.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33748/0 (socket says 172.21.15.38:33748) 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 -- 172.21.15.38:0/2038758344 learned_addr learned my addr 172.21.15.38:0/2038758344 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45840ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45840120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 -- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 msgr2=0x562e45840ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45840ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 -- 172.21.15.38:0/2038758344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 msgr2=0x562e45840120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45840120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 -- 172.21.15.38:0/2038758344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e454ac580 con 0x562e45838980 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45840120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb9900115a0 tx=0x7fb990009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31acc9b7c72c2d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.137+0000 7fb988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:32.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990013670 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb990013810 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb998004520 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb990026050 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e45576960 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998004520 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb99001d400 con 0x562e45838980 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 10 client.15810 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 20 client.15810 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e45852e10 0x562e45855240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 4 client.15810 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.138+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e457aab90 con 0x562e45852e10 2024-09-19T05:49:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.139+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e45852e10 0x562e45855240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.139+0000 7fb988ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.139+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e45852e10 0x562e45855240 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ab3592cde8bcc7 server_cookie=ca1e9bbd168e1b72 in_seq=0 out_seq=0 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb988ff9640 1 -- 172.21.15.38:0/2038758344 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e457aab90 con 0x562e45852e10 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb988ff9640 10 client.15810 handle_command_reply: tid=0 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 client.15810 shutdown 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 msgr2=0x562e45840660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb9900115a0 tx=0x7fb990009f90 comp rx=0 tx=0).stop 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 shutdown_connections 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e45852e10 0x562e45855240 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45843240 0x562e45840ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838980 0x562e45840660 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2038758344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838060 0x562e45840120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 >> 172.21.15.38:0/2038758344 conn(0x562e458345f0 msgr2=0x562e45833750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb988ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 shutdown_connections 2024-09-19T05:49:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2038758344 wait complete. 2024-09-19T05:49:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.140+0000 7fb9a9c05740 20 client.15810 trim_cache size 0 max 16384 2024-09-19T05:49:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 Processor -- start 2024-09-19T05:49:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 -- start start 2024-09-19T05:49:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838980 0x562e45833640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 0x562e4585ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01aed40 con 0x562e45838060 2024-09-19T05:49:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b0880 con 0x562e45843240 2024-09-19T05:49:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01af410 con 0x562e45838980 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33762/0 (socket says 172.21.15.38:33762) 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 -- 172.21.15.38:0/2424759055 learned_addr learned my addr 172.21.15.38:0/2424759055 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838980 0x562e45833640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 0x562e4585ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 -- 172.21.15.38:0/2424759055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838980 msgr2=0x562e45833640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838980 0x562e45833640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 -- 172.21.15.38:0/2424759055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 msgr2=0x562e4585ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 0x562e4585ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a55c8640 1 -- 172.21.15.38:0/2424759055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e45842810 con 0x562e45838060 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.158+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 0x562e4585ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007590 tx=0x7fb99c0ccf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f063b48a824e62c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb98affd640 1 -- 172.21.15.38:0/2424759055 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0b3020 con 0x562e45838060 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb98affd640 1 -- 172.21.15.38:0/2424759055 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c09c250 con 0x562e45838060 2024-09-19T05:49:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb98affd640 1 -- 172.21.15.38:0/2424759055 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0d5020 con 0x562e45838060 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2424759055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 msgr2=0x562e4583ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007590 tx=0x7fb99c0ccf00 comp rx=0 tx=0).stop 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2424759055 shutdown_connections 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45843240 0x562e4585ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838980 0x562e45833640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.159+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/2424759055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45838060 0x562e4583ab10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2424759055 >> 172.21.15.38:0/2424759055 conn(0x562e458345f0 msgr2=0x562e45832f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2424759055 shutdown_connections 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- 172.21.15.38:0/2424759055 wait complete. 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- start start 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 Processor -- start 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838060 0x562e45847370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838980 0x562e458478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b01b0 con 0x562e45843240 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01afae0 con 0x562e45838980 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01acd20 con 0x562e45838060 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.160+0000 7fb9a9c05740 10 client.0 _ll_register_callbacks cb 0x7fb990009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:49:32.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33774/0 (socket says 172.21.15.38:33774) 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/998509814 learned_addr learned my addr 172.21.15.38:0/998509814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838060 0x562e45847370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a4dc7640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838980 0x562e458478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838060 msgr2=0x562e45847370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838060 0x562e45847370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/998509814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838980 msgr2=0x562e458478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838980 0x562e458478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 -- 172.21.15.38:0/998509814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e458bd060 con 0x562e45843240 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a5dc9640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb998046690 tx=0x7fb998003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174ed08b91800462 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980098d0 con 0x562e45843240 2024-09-19T05:49:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb99802f600 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99c09c250 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb998065050 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a9c05740 10 client.15816 fetch_fsmap learned FSMap version 28 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.161+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e458be1f0 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c09c250 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb998030600 con 0x562e45843240 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 10 client.15816 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 20 client.15816 populate_metadata read hostname 'smithi038' 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e458be520 0x562e458c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 4 client.15816 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "814baed3", "release": true, "await": true}] 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "814baed3", "release": true, "await": true}) -- 0x562e457a50a0 con 0x562e458be520 2024-09-19T05:49:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.162+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e458be520 0x562e458c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.163+0000 7fb9a55c8640 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e458be520 0x562e458c0910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c688ae3fd4b038fd server_cookie=94ff5cf23e6359fa in_seq=0 out_seq=0 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.163+0000 7fb988ff9640 10 client.15816 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.165+0000 7fb988ff9640 1 -- 172.21.15.38:0/998509814 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562e457a50a0 con 0x562e458be520 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.165+0000 7fb988ff9640 10 client.15816 handle_command_reply: tid=0 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 client.15816 shutdown 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 msgr2=0x562e458bcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb998046690 tx=0x7fb998003f90 comp rx=0 tx=0).stop 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 shutdown_connections 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x562e458be520 0x562e458c0910 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e45843240 0x562e458bcb20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e45838980 0x562e458478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 --2- 172.21.15.38:0/998509814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562e45838060 0x562e45847370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 >> 172.21.15.38:0/998509814 conn(0x562e458345f0 msgr2=0x562e45832f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb988ff9640 0 client.15816 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 shutdown_connections 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 1 -- 172.21.15.38:0/998509814 wait complete. 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.166+0000 7fb9a9c05740 20 client.15816 trim_cache size 0 max 16384 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 32, 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "814baed3": { 2024-09-19T05:49:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 32, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 145.8, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:49:32.175 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:49:32.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 msgr2=0x7fb98c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 0x7fb98c0a5700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb99804d000 tx=0x7fb99804a000 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 msgr2=0x7fb9a0199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c890 tx=0x7fb99c00cc90 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 shutdown_connections 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a0102ef0 0x7fb9a01a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9a0102570 0x7fb9a0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 --2- 172.21.15.38:0/956695928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0100e20 0x7fb9a0199330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 >> 172.21.15.38:0/956695928 conn(0x7fb9a00fc7d0 msgr2=0x7fb9a0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 shutdown_connections 2024-09-19T05:49:32.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:49:32.167+0000 7fb9a782c640 1 -- 172.21.15.38:0/956695928 wait complete. 2024-09-19T05:49:32.905 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '814baed3', total seconds elapsed: 29.0 2024-09-19T05:49:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:33 smithi038 ceph-mon[31227]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:49:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:33 smithi179 ceph-mon[35781]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:49:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:33 smithi047 ceph-mon[35327]: pgmap v602: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:49:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:35 smithi038 ceph-mon[31227]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:35 smithi179 ceph-mon[35781]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:35.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:35 smithi047 ceph-mon[35327]: pgmap v603: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:37 smithi038 ceph-mon[31227]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:37 smithi179 ceph-mon[35781]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:37.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:37 smithi047 ceph-mon[35327]: pgmap v604: 129 pgs: 129 active+clean; 3.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-19T05:49:39.221 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:189/600: open fd = 3 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:190/600: open fd = 3 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:191/600: open fd = 3 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.222 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:192/600: open fd = 3 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:193/600: open fd = 3 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:194/600: open fd = 3 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.223 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:195/600: open fd = 3 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:196/600: open fd = 3 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:197/600: open fd = 3 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.224 INFO:tasks.workunit.client.0.smithi038.stdout:198/600: open fd = 3 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:199/600: open fd = 3 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:200/600: open fd = 3 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:201/600: open fd = 3 2024-09-19T05:49:39.225 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:202/600: open fd = 3 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:203/600: open fd = 3 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.226 INFO:tasks.workunit.client.0.smithi038.stdout:204/600: open fd = 3 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:205/600: open fd = 3 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:206/600: open fd = 3 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.227 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:207/600: open fd = 3 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:208/600: open fd = 3 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.228 INFO:tasks.workunit.client.0.smithi038.stdout:209/600: open fd = 3 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:210/600: open fd = 3 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:211/600: open fd = 3 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:212/600: open fd = 3 2024-09-19T05:49:39.229 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:213/600: open fd = 3 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:214/600: open fd = 3 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:215/600: open fd = 3 2024-09-19T05:49:39.230 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:216/600: open fd = 3 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:217/600: open fd = 3 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:218/600: open fd = 3 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.231 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:219/600: open fd = 3 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:220/600: open fd = 3 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:221/600: open fd = 3 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.232 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:222/600: open fd = 3 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:223/600: open fd = 3 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:224/600: open fd = 3 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.233 INFO:tasks.workunit.client.0.smithi038.stdout:225/600: open fd = 3 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:226/600: open fd = 3 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:227/600: open fd = 3 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:228/600: open fd = 3 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.234 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:229/600: open fd = 3 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:230/600: open fd = 3 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:231/600: open fd = 3 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:232/600: open fd = 3 2024-09-19T05:49:39.235 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:233/600: open fd = 3 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:234/600: open fd = 3 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:235/600: open fd = 3 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.236 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:236/600: open fd = 3 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:237/600: open fd = 3 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:238/600: open fd = 3 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.237 INFO:tasks.workunit.client.0.smithi038.stdout:239/600: open fd = 3 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:240/600: open fd = 3 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:241/600: open fd = 3 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:242/600: open fd = 3 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.238 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:243/600: open fd = 3 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:244/600: open fd = 3 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:245/600: open fd = 3 2024-09-19T05:49:39.239 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:246/600: open fd = 3 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:247/600: open fd = 3 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.240 INFO:tasks.workunit.client.0.smithi038.stdout:248/600: open fd = 3 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:249/600: open fd = 3 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:250/600: open fd = 3 2024-09-19T05:49:39.241 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.242 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:49:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:39 smithi038 ceph-mon[31227]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:49:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:39 smithi179 ceph-mon[35781]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:49:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:39 smithi047 ceph-mon[35327]: pgmap v605: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2024-09-19T05:49:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:41 smithi038 ceph-mon[31227]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:49:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:41 smithi179 ceph-mon[35781]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:49:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:41 smithi047 ceph-mon[35327]: pgmap v606: 129 pgs: 129 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:49:43.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:43 smithi038 ceph-mon[31227]: pgmap v607: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T05:49:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:43 smithi179 ceph-mon[35781]: pgmap v607: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T05:49:43.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:43 smithi047 ceph-mon[35327]: pgmap v607: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-19T05:49:45.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:45 smithi038 ceph-mon[31227]: pgmap v608: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:45 smithi179 ceph-mon[35781]: pgmap v608: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:45.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:45 smithi047 ceph-mon[35327]: pgmap v608: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:47 smithi038 ceph-mon[31227]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:49:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:47 smithi179 ceph-mon[35781]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:49:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:47 smithi047 ceph-mon[35327]: pgmap v609: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:49:49.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:49 smithi038 ceph-mon[31227]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:49 smithi179 ceph-mon[35781]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:49.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:49 smithi047 ceph-mon[35327]: pgmap v610: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:51 smithi038 ceph-mon[31227]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:51 smithi179 ceph-mon[35781]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:51 smithi047 ceph-mon[35327]: pgmap v611: 129 pgs: 129 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:49:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:53 smithi038 ceph-mon[31227]: pgmap v612: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:53 smithi179 ceph-mon[35781]: pgmap v612: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:53.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:53 smithi047 ceph-mon[35327]: pgmap v612: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:49:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:55 smithi179 ceph-mon[35781]: pgmap v613: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:49:55.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:55 smithi047 ceph-mon[35327]: pgmap v613: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:49:56.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:55 smithi038 ceph-mon[31227]: pgmap v613: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:49:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:57 smithi179 ceph-mon[35781]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:49:57.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:57 smithi047 ceph-mon[35327]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:49:58.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:57 smithi038 ceph-mon[31227]: pgmap v614: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:49:58.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:49:58 smithi179 ceph-mon[35781]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:49:58.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:49:58 smithi047 ceph-mon[35327]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:49:59.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:49:58 smithi038 ceph-mon[31227]: pgmap v615: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:00 smithi038 ceph-mon[31227]: pgmap v616: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:01.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:00 smithi038 ceph-mon[31227]: overall HEALTH_OK 2024-09-19T05:50:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:00 smithi179 ceph-mon[35781]: pgmap v616: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:00 smithi179 ceph-mon[35781]: overall HEALTH_OK 2024-09-19T05:50:01.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:00 smithi047 ceph-mon[35327]: pgmap v616: 129 pgs: 129 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:01.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:00 smithi047 ceph-mon[35327]: overall HEALTH_OK 2024-09-19T05:50:03.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:02 smithi038 ceph-mon[31227]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:03.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:02 smithi179 ceph-mon[35781]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:03.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:02 smithi047 ceph-mon[35327]: pgmap v617: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:04 smithi038 ceph-mon[31227]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:05.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:04 smithi179 ceph-mon[35781]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:05.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:04 smithi047 ceph-mon[35327]: pgmap v618: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:06 smithi038 ceph-mon[31227]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:06 smithi179 ceph-mon[35781]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:07.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:06 smithi047 ceph-mon[35327]: pgmap v619: 129 pgs: 129 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:09.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:08 smithi038 ceph-mon[31227]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:08 smithi179 ceph-mon[35781]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:09.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:08 smithi047 ceph-mon[35327]: pgmap v620: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:10.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:09 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:50:10.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:09 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:50:10.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:09 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:50:11.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:10 smithi038 ceph-mon[31227]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:11.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:50:11.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:50:11.145 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:50:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:10 smithi179 ceph-mon[35781]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:50:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:50:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:50:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:10 smithi047 ceph-mon[35327]: pgmap v621: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:50:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:50:11.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:50:13.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:12 smithi038 ceph-mon[31227]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:13.201 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:12 smithi179 ceph-mon[35781]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:13.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:12 smithi047 ceph-mon[35327]: pgmap v622: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:15.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:14 smithi179 ceph-mon[35781]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:15.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:14 smithi047 ceph-mon[35327]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:14 smithi038 ceph-mon[31227]: pgmap v623: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:17.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:16 smithi179 ceph-mon[35781]: pgmap v624: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:17.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:16 smithi047 ceph-mon[35327]: pgmap v624: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:17.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:16 smithi038 ceph-mon[31227]: pgmap v624: 129 pgs: 129 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:19.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:18 smithi179 ceph-mon[35781]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:19.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:18 smithi047 ceph-mon[35327]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:18 smithi038 ceph-mon[31227]: pgmap v625: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:21.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:20 smithi179 ceph-mon[35781]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:21.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:20 smithi047 ceph-mon[35327]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:21.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:20 smithi038 ceph-mon[31227]: pgmap v626: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:23.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:22 smithi179 ceph-mon[35781]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:23.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:22 smithi047 ceph-mon[35327]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:23.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:22 smithi038 ceph-mon[31227]: pgmap v627: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:24 smithi179 ceph-mon[35781]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:25.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:24 smithi047 ceph-mon[35327]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:25.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:24 smithi038 ceph-mon[31227]: pgmap v628: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:27.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:26 smithi179 ceph-mon[35781]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:27.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:26 smithi047 ceph-mon[35327]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:27.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:26 smithi038 ceph-mon[31227]: pgmap v629: 129 pgs: 129 active+clean; 4.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:29.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:28 smithi179 ceph-mon[35781]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:29.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:28 smithi047 ceph-mon[35327]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:29.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:28 smithi038 ceph-mon[31227]: pgmap v630: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:31.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:30 smithi047 ceph-mon[35327]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:30 smithi038 ceph-mon[31227]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:30 smithi179 ceph-mon[35781]: pgmap v631: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:33.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:32 smithi047 ceph-mon[35327]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:33.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:32 smithi038 ceph-mon[31227]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:32 smithi179 ceph-mon[35781]: pgmap v632: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:35.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:34 smithi047 ceph-mon[35327]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:34 smithi038 ceph-mon[31227]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:34 smithi179 ceph-mon[35781]: pgmap v633: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:37.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:36 smithi047 ceph-mon[35327]: pgmap v634: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:50:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:36 smithi038 ceph-mon[31227]: pgmap v634: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:50:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:36 smithi179 ceph-mon[35781]: pgmap v634: 129 pgs: 129 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:50:39.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:38 smithi047 ceph-mon[35327]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:50:39.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:38 smithi038 ceph-mon[31227]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:50:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:38 smithi179 ceph-mon[35781]: pgmap v635: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:50:41.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:40 smithi047 ceph-mon[35327]: pgmap v636: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:41.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:40 smithi038 ceph-mon[31227]: pgmap v636: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:41.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:40 smithi179 ceph-mon[35781]: pgmap v636: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:50:41.697 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.697 INFO:tasks.workunit.client.0.smithi038.stdout:251/600: open fd = 3 2024-09-19T05:50:41.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:252/600: open fd = 3 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:253/600: open fd = 3 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:254/600: open fd = 3 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:255/600: open fd = 3 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:256/600: open fd = 3 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:257/600: open fd = 3 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.699 INFO:tasks.workunit.client.0.smithi038.stdout:258/600: open fd = 3 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:259/600: open fd = 3 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:260/600: open fd = 3 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:261/600: open fd = 3 2024-09-19T05:50:41.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:262/600: open fd = 3 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:263/600: open fd = 3 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:264/600: open fd = 3 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:265/600: open fd = 3 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:266/600: open fd = 3 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:267/600: open fd = 3 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.702 INFO:tasks.workunit.client.0.smithi038.stdout:268/600: open fd = 3 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:269/600: open fd = 3 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:270/600: open fd = 3 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:271/600: open fd = 3 2024-09-19T05:50:41.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:272/600: open fd = 3 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:273/600: open fd = 3 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:274/600: open fd = 3 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.704 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:275/600: open fd = 3 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:276/600: open fd = 3 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:277/600: open fd = 3 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.705 INFO:tasks.workunit.client.0.smithi038.stdout:278/600: open fd = 3 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:279/600: open fd = 3 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:280/600: open fd = 3 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:281/600: open fd = 3 2024-09-19T05:50:41.706 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:282/600: open fd = 3 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:283/600: open fd = 3 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:284/600: open fd = 3 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.707 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:285/600: open fd = 3 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:286/600: open fd = 3 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:287/600: open fd = 3 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:288/600: open fd = 3 2024-09-19T05:50:41.708 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:289/600: open fd = 3 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:290/600: open fd = 3 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:291/600: open fd = 3 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.709 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:292/600: open fd = 3 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:293/600: open fd = 3 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:294/600: open fd = 3 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.710 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:295/600: open fd = 3 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:296/600: open fd = 3 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.711 INFO:tasks.workunit.client.0.smithi038.stdout:297/600: open fd = 3 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:298/600: open fd = 3 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:299/600: open fd = 3 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:300/600: open fd = 3 2024-09-19T05:50:41.712 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:301/600: open fd = 3 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:302/600: open fd = 3 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:303/600: open fd = 3 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.713 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:304/600: open fd = 3 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:305/600: open fd = 3 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:306/600: open fd = 3 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.714 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:307/600: open fd = 3 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:308/600: open fd = 3 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:309/600: open fd = 3 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:310/600: open fd = 3 2024-09-19T05:50:41.715 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:311/600: open fd = 3 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:312/600: open fd = 3 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:41.716 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:50:43.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:42 smithi038 ceph-mon[31227]: pgmap v637: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:42 smithi179 ceph-mon[35781]: pgmap v637: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:43.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:42 smithi047 ceph-mon[35327]: pgmap v637: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:50:45.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:44 smithi038 ceph-mon[31227]: pgmap v638: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:44 smithi179 ceph-mon[35781]: pgmap v638: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:44 smithi047 ceph-mon[35327]: pgmap v638: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:47.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:46 smithi038 ceph-mon[31227]: pgmap v639: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:46 smithi179 ceph-mon[35781]: pgmap v639: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:47.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:46 smithi047 ceph-mon[35327]: pgmap v639: 129 pgs: 129 active+clean; 4.2 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:50:49.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:48 smithi038 ceph-mon[31227]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:48 smithi179 ceph-mon[35781]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:48 smithi047 ceph-mon[35327]: pgmap v640: 129 pgs: 129 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:51.277 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:51 smithi047 ceph-mon[35327]: pgmap v641: 129 pgs: 129 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:51.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:51 smithi038 ceph-mon[31227]: pgmap v641: 129 pgs: 129 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:51 smithi179 ceph-mon[35781]: pgmap v641: 129 pgs: 129 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:53.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:53 smithi038 ceph-mon[31227]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:53 smithi179 ceph-mon[35781]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:53.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:53 smithi047 ceph-mon[35327]: pgmap v642: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:55.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:55 smithi038 ceph-mon[31227]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:55.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:55 smithi179 ceph-mon[35781]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:55.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:55 smithi047 ceph-mon[35327]: pgmap v643: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:57.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:57 smithi038 ceph-mon[31227]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:57 smithi179 ceph-mon[35781]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:57.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:57 smithi047 ceph-mon[35327]: pgmap v644: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:50:59.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:50:59 smithi038 ceph-mon[31227]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:59.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:50:59 smithi179 ceph-mon[35781]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:50:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:50:59 smithi047 ceph-mon[35327]: pgmap v645: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:51:01.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:01 smithi038 ceph-mon[31227]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:01.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:01 smithi179 ceph-mon[35781]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:01.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:01 smithi047 ceph-mon[35327]: pgmap v646: 129 pgs: 129 active+clean; 4.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:03.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:03 smithi038 ceph-mon[31227]: pgmap v647: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:51:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:03 smithi179 ceph-mon[35781]: pgmap v647: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:51:03.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:03 smithi047 ceph-mon[35327]: pgmap v647: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:51:05.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:05 smithi038 ceph-mon[31227]: pgmap v648: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:05 smithi179 ceph-mon[35781]: pgmap v648: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:05.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:05 smithi047 ceph-mon[35327]: pgmap v648: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:07.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:07 smithi038 ceph-mon[31227]: pgmap v649: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:51:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:07 smithi179 ceph-mon[35781]: pgmap v649: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:51:07.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:07 smithi047 ceph-mon[35327]: pgmap v649: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:51:08.548 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:51:08.549 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T05:51:08.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:51:08.824 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:09.240 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:09 smithi038 ceph-mon[31227]: pgmap v650: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.321+0000 7fe983ff0640 1 -- 172.21.15.38:0/3836563619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 msgr2=0x7fe97c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.321+0000 7fe983ff0640 1 --2- 172.21.15.38:0/3836563619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c1021c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe978009900 tx=0x7fe97802a410 comp rx=0 tx=0).stop 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 -- 172.21.15.38:0/3836563619 shutdown_connections 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 --2- 172.21.15.38:0/3836563619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c106f10 0x7fe97c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 --2- 172.21.15.38:0/3836563619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c102790 0x7fe97c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 --2- 172.21.15.38:0/3836563619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c1021c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 -- 172.21.15.38:0/3836563619 >> 172.21.15.38:0/3836563619 conn(0x7fe97c0fd8c0 msgr2=0x7fe97c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 -- 172.21.15.38:0/3836563619 shutdown_connections 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 -- 172.21.15.38:0/3836563619 wait complete. 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 Processor -- start 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.322+0000 7fe983ff0640 1 -- start start 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c102790 0x7fe97c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 0x7fe97c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c111f90 con 0x7fe97c101de0 2024-09-19T05:51:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c111e10 con 0x7fe97c102790 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c112110 con 0x7fe97c106f10 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41202/0 (socket says 172.21.15.38:41202) 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 -- 172.21.15.38:0/1267497536 learned_addr learned my addr 172.21.15.38:0/1267497536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe98158b640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c102790 0x7fe97c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe98258d640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 0x7fe97c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 -- 172.21.15.38:0/1267497536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 msgr2=0x7fe97c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 0x7fe97c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 -- 172.21.15.38:0/1267497536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c102790 msgr2=0x7fe97c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c102790 0x7fe97c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 -- 172.21.15.38:0/1267497536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c19d440 con 0x7fe97c101de0 2024-09-19T05:51:09.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe98258d640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 0x7fe97c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe981d8c640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe978006010 tx=0x7fe9780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97803f030 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978003ce0 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978035070 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c19d6d0 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.323+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1a5010 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.325+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97c1032c0 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.326+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe978042030 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.326+0000 7fe972ffd640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 0x7fe9680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.326+0000 7fe98158b640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 0x7fe9680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.326+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe9780f00f0 con 0x7fe97c101de0 2024-09-19T05:51:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.327+0000 7fe98158b640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 0x7fe9680a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0fdd50 tx=0x7fe96c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:09.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.330+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe9780bca50 con 0x7fe97c101de0 2024-09-19T05:51:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:09 smithi179 ceph-mon[35781]: pgmap v650: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:51:09.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:09 smithi047 ceph-mon[35327]: pgmap v650: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:51:09.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.543+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe97c10a0f0 con 0x7fe97c101de0 2024-09-19T05:51:09.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.544+0000 7fe972ffd640 1 -- 172.21.15.38:0/1267497536 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fe9780bc350 con 0x7fe97c101de0 2024-09-19T05:51:09.546 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:51:09.546 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:51:09.547 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 msgr2=0x7fe9680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 0x7fe9680a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0fdd50 tx=0x7fe96c009110 comp rx=0 tx=0).stop 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 msgr2=0x7fe97c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe978006010 tx=0x7fe9780042f0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 shutdown_connections 2024-09-19T05:51:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe9680a3240 0x7fe9680a5700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe97c106f10 0x7fe97c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe97c102790 0x7fe97c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 --2- 172.21.15.38:0/1267497536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe97c101de0 0x7fe97c198490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 >> 172.21.15.38:0/1267497536 conn(0x7fe97c0fd8c0 msgr2=0x7fe97c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 shutdown_connections 2024-09-19T05:51:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:09.546+0000 7fe983ff0640 1 -- 172.21.15.38:0/1267497536 wait complete. 2024-09-19T05:51:10.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:10 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1267497536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:10.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:10 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:51:10.340 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T05:51:10.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T05:51:10.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:10 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1267497536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:10.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:10 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:51:10.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:10 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1267497536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:10.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:10 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:51:11.247 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:11.371 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:11 smithi038 ceph-mon[31227]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:11 smithi179 ceph-mon[35781]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:11.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:11 smithi047 ceph-mon[35327]: pgmap v651: 129 pgs: 129 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 -- 172.21.15.38:0/2972346309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4100e20 msgr2=0x7efec4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2972346309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4100e20 0x7efec4101200 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efebc009ca0 tx=0x7efebc02a950 comp rx=0 tx=0).stop 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 -- 172.21.15.38:0/2972346309 shutdown_connections 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2972346309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 0x7efec410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2972346309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4102570 0x7efec41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2972346309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4100e20 0x7efec4101200 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 -- 172.21.15.38:0/2972346309 >> 172.21.15.38:0/2972346309 conn(0x7efec40fc7d0 msgr2=0x7efec40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 -- 172.21.15.38:0/2972346309 shutdown_connections 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.790+0000 7efec8b2a640 1 -- 172.21.15.38:0/2972346309 wait complete. 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 Processor -- start 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 -- start start 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 0x7efec4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 0x7efec4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efec4110f80 con 0x7efec4102570 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efec4110e00 con 0x7efec4100e20 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.791+0000 7efec8b2a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efec4111100 con 0x7efec4102ef0 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 0x7efec4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:54028/0 (socket says 172.21.15.38:54028) 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec259c640 1 -- 172.21.15.38:0/2572525500 learned_addr learned my addr 172.21.15.38:0/2572525500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec2d9d640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 0x7efec4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 -- 172.21.15.38:0/2572525500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 msgr2=0x7efec4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 0x7efec4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 -- 172.21.15.38:0/2572525500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 msgr2=0x7efec4197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 -- 172.21.15.38:0/2572525500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41a69c0 con 0x7efec4102570 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec259c640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec1d9b640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 0x7efec4197690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9f0 tx=0x7efeac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efebb7fe640 1 -- 172.21.15.38:0/2572525500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00ccc0 con 0x7efec4102570 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efebb7fe640 1 -- 172.21.15.38:0/2572525500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac010910 con 0x7efec4102570 2024-09-19T05:51:11.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a6cb0 con 0x7efec4102570 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.792+0000 7efebb7fe640 1 -- 172.21.15.38:0/2572525500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac018960 con 0x7efec4102570 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.793+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a71e0 con 0x7efec4102570 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 Processor -- start 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 -- start start 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efebb7fe640 1 -- 172.21.15.38:0/2572525500 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7efeac010ac0 con 0x7efec4102570 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f364b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 0x55ed6f371d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f2ca2a0 con 0x55ed6f365080 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f271b10 con 0x55ed6f36f940 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efecaf03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f2c6260 con 0x55ed6f364760 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efec259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efec259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41228/0 (socket says 172.21.15.38:41228) 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efec259c640 1 -- 172.21.15.38:0/4233512654 learned_addr learned my addr 172.21.15.38:0/4233512654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.796+0000 7efec2d9d640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f364b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 -- 172.21.15.38:0/4233512654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 msgr2=0x55ed6f364b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f364b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 -- 172.21.15.38:0/4233512654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 msgr2=0x55ed6f371d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec1d9b640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 0x55ed6f371d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 0x55ed6f371d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 -- 172.21.15.38:0/4233512654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6f03bb90 con 0x55ed6f365080 2024-09-19T05:51:11.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efebb7fe640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 0x7efeb00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec1d9b640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 0x55ed6f371d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec2d9d640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 0x7efeb00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efebb7fe640 1 -- 172.21.15.38:0/2572525500 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7efeac0cffe0 con 0x7efec4102570 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efec259c640 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efebc005f80 tx=0x7efebc02fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72db1d68d364f969 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efeb8ff9640 1 -- 172.21.15.38:0/4233512654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc048070 con 0x55ed6f365080 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efeb8ff9640 1 -- 172.21.15.38:0/4233512654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc003e50 con 0x55ed6f365080 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.797+0000 7efeb8ff9640 1 -- 172.21.15.38:0/4233512654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc034040 con 0x55ed6f365080 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 -- 172.21.15.38:0/4233512654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 msgr2=0x55ed6f36f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efec2d9d640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 0x7efeb00a5780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb4005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efebc005f80 tx=0x7efebc02fe00 comp rx=0 tx=0).stop 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 -- 172.21.15.38:0/4233512654 shutdown_connections 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f36f940 0x55ed6f371d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36f400 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 --2- 172.21.15.38:0/4233512654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f364b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.798+0000 7efecaf03740 1 -- 172.21.15.38:0/4233512654 >> 172.21.15.38:0/4233512654 conn(0x55ed6f360cf0 msgr2=0x55ed6f3610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- 172.21.15.38:0/4233512654 shutdown_connections 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- 172.21.15.38:0/4233512654 wait complete. 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- start start 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 Processor -- start 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f36f940 0x55ed6f36d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f372510 con 0x55ed6f365080 2024-09-19T05:51:11.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f2526e0 con 0x55ed6f364760 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.799+0000 7efecaf03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed6f372690 con 0x55ed6f36f940 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efecaf03740 10 client.0 _ll_register_callbacks cb 0x55ed6f276a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:54046/0 (socket says 172.21.15.38:54046) 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 -- 172.21.15.38:0/3203336830 learned_addr learned my addr 172.21.15.38:0/3203336830 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec1d9b640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41236/0 (socket says 172.21.15.38:41236) 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec2d9d640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f36f940 0x55ed6f36d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 -- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f36f940 msgr2=0x55ed6f36d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f36f940 0x55ed6f36d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 -- 172.21.15.38:0/3203336830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 msgr2=0x55ed6f36ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 -- 172.21.15.38:0/3203336830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed6f37b9e0 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efec259c640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efebc02fe70 tx=0x7efebc02f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de79bfca9c7fba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc051060 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efebc003e50 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.800+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7efebc040040 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efebc058050 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 10 client.25505 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed6f37e2c0 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc003e50 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7efebc030960 con 0x55ed6f364760 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 10 client.25505 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:11.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 10 client.25505 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 20 client.25505 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f37f610 0x55ed6f381a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 4 client.25505 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed6f2d79d0 con 0x55ed6f37f610 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.801+0000 7efec1d9b640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f37f610 0x55ed6f381a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.802+0000 7efea9ffb640 10 client.25505 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.802+0000 7efec1d9b640 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f37f610 0x55ed6f381a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238118e1530cfbc1 server_cookie=35a6889bba0d956 in_seq=0 out_seq=0 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.802+0000 7efea9ffb640 1 -- 172.21.15.38:0/3203336830 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed6f2d79d0 con 0x55ed6f37f610 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.802+0000 7efea9ffb640 10 client.25505 handle_command_reply: tid=0 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.802+0000 7efecaf03740 1 client.25505 shutdown 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 msgr2=0x55ed6f36c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efebc02fe70 tx=0x7efebc02f890 comp rx=0 tx=0).stop 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 shutdown_connections 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f37f610 0x55ed6f381a00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f36f940 0x55ed6f36d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efea9ffb640 0 client.25505 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f365080 0x55ed6f36ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 --2- 172.21.15.38:0/3203336830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f36c790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 >> 172.21.15.38:0/3203336830 conn(0x55ed6f360cf0 msgr2=0x55ed6f3669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 shutdown_connections 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 1 -- 172.21.15.38:0/3203336830 wait complete. 2024-09-19T05:51:11.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.803+0000 7efecaf03740 20 client.25505 trim_cache size 0 max 16384 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.821+0000 7efecaf03740 1 Processor -- start 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.821+0000 7efecaf03740 1 -- start start 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f386fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 0x55ed6f3874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efec41b0ee0 con 0x55ed6f36f940 2024-09-19T05:51:11.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efec41b2a20 con 0x55ed6f365080 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efecaf03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efec41b15b0 con 0x55ed6f364760 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41240/0 (socket says 172.21.15.38:41240) 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 -- 172.21.15.38:0/996652564 learned_addr learned my addr 172.21.15.38:0/996652564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec1d9b640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 0x55ed6f3874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec259c640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f386fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 -- 172.21.15.38:0/996652564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 msgr2=0x55ed6f386fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f386fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 -- 172.21.15.38:0/996652564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 msgr2=0x55ed6f3874e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 0x55ed6f3874e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 -- 172.21.15.38:0/996652564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed6f360b20 con 0x55ed6f36f940 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec1d9b640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 0x55ed6f3874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:51:11.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.822+0000 7efec2d9d640 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efeb4003f20 tx=0x7efeb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6832f9326fb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efeabfff640 1 -- 172.21.15.38:0/996652564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4003f50 con 0x55ed6f36f940 2024-09-19T05:51:11.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efeabfff640 1 -- 172.21.15.38:0/996652564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb4029960 con 0x55ed6f36f940 2024-09-19T05:51:11.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efeabfff640 1 -- 172.21.15.38:0/996652564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4011960 con 0x55ed6f36f940 2024-09-19T05:51:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 -- 172.21.15.38:0/996652564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 msgr2=0x55ed6f369a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efeb4003f20 tx=0x7efeb4019000 comp rx=0 tx=0).stop 2024-09-19T05:51:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 -- 172.21.15.38:0/996652564 shutdown_connections 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f369a10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f365080 0x55ed6f3874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 --2- 172.21.15.38:0/996652564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f364760 0x55ed6f386fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.823+0000 7efecaf03740 1 -- 172.21.15.38:0/996652564 >> 172.21.15.38:0/996652564 conn(0x55ed6f360cf0 msgr2=0x55ed6f3669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- 172.21.15.38:0/996652564 shutdown_connections 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- 172.21.15.38:0/996652564 wait complete. 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- start start 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 Processor -- start 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f365080 0x55ed6f377e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f374910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efec41b2350 con 0x55ed6f36f940 2024-09-19T05:51:11.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efec41b1c80 con 0x55ed6f364760 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efec41aeec0 con 0x55ed6f365080 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efecaf03740 10 client.0 _ll_register_callbacks cb 0x7efebc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efec259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:54056/0 (socket says 172.21.15.38:54056) 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efec1d9b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f365080 0x55ed6f377e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efec259c640 1 -- 172.21.15.38:0/2585772272 learned_addr learned my addr 172.21.15.38:0/2585772272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.824+0000 7efec2d9d640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f374910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 -- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f365080 msgr2=0x55ed6f377e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f365080 0x55ed6f377e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 -- 172.21.15.38:0/2585772272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 msgr2=0x55ed6f374910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f374910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 -- 172.21.15.38:0/2585772272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed6f374e50 con 0x55ed6f364760 2024-09-19T05:51:11.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec2d9d640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f374910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efec259c640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efebc005fa0 tx=0x7efebc02fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ff086c290b0cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc051040 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7efebc049040 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb4029960 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efebc048030 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efecaf03740 10 client.25511 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.825+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed6f3ea620 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb4029960 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7efebc03a430 con 0x55ed6f364760 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 10 client.25511 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 10 client.25511 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:11.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 20 client.25511 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f3ead50 0x55ed6f3ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 4 client.25511 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55ed6f2d1d20 con 0x55ed6f3ead50 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.826+0000 7efec1d9b640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f3ead50 0x55ed6f3ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.827+0000 7efec1d9b640 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f3ead50 0x55ed6f3ed170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ff320e20624731 server_cookie=d30df323a576dbbd in_seq=0 out_seq=0 2024-09-19T05:51:11.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.827+0000 7efea9ffb640 10 client.25511 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:11.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efea9ffb640 1 -- 172.21.15.38:0/2585772272 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ed6f2d1d20 con 0x55ed6f3ead50 2024-09-19T05:51:11.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efea9ffb640 10 client.25511 handle_command_reply: tid=0 2024-09-19T05:51:11.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 client.25511 shutdown 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 msgr2=0x55ed6f377940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efebc005fa0 tx=0x7efebc02fe00 comp rx=0 tx=0).stop 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 shutdown_connections 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55ed6f3ead50 0x55ed6f3ed170 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ed6f36f940 0x55ed6f374910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed6f365080 0x55ed6f377e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efea9ffb640 0 client.25511 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 --2- 172.21.15.38:0/2585772272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed6f364760 0x55ed6f377940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.835+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 >> 172.21.15.38:0/2585772272 conn(0x55ed6f360cf0 msgr2=0x55ed6f35f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 shutdown_connections 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efecaf03740 1 -- 172.21.15.38:0/2585772272 wait complete. 2024-09-19T05:51:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efecaf03740 20 client.25511 trim_cache size 0 max 16384 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 34, 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "769a986b": { 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 34, 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:51:11.839 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.0, 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:11.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 msgr2=0x7efeb00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 0x7efeb00a5780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb4005cc0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 msgr2=0x7efec4197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:11.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 0x7efec4197690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9f0 tx=0x7efeac00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 shutdown_connections 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7efeb00a32c0 0x7efeb00a5780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4102ef0 0x7efec4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efec4102570 0x7efec4197690 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 --2- 172.21.15.38:0/2572525500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec4100e20 0x7efec4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.836+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 >> 172.21.15.38:0/2572525500 conn(0x7efec40fc7d0 msgr2=0x7efec40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:11.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.837+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 shutdown_connections 2024-09-19T05:51:11.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:11.837+0000 7efec8b2a640 1 -- 172.21.15.38:0/2572525500 wait complete. 2024-09-19T05:51:12.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:12 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.747 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:12 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.778 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '769a986b', quiesce took 4.2 seconds. Will release after: 25.8 2024-09-19T05:51:12.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:12.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:12 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: pgmap v652: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:51:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: pgmap v652: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:51:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:51:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:51:13.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: pgmap v652: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-09-19T05:51:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:51:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:51:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:51:15.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:15 smithi038 ceph-mon[31227]: pgmap v653: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:51:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:15 smithi179 ceph-mon[35781]: pgmap v653: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:51:15.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:15 smithi047 ceph-mon[35327]: pgmap v653: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:51:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:17 smithi038 ceph-mon[31227]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:17 smithi179 ceph-mon[35781]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:17.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:17 smithi047 ceph-mon[35327]: pgmap v654: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:51:19.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:19 smithi038 ceph-mon[31227]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-19T05:51:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:19 smithi179 ceph-mon[35781]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-19T05:51:19.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:19 smithi047 ceph-mon[35327]: pgmap v655: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-09-19T05:51:21.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:21 smithi038 ceph-mon[31227]: pgmap v656: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:21 smithi179 ceph-mon[35781]: pgmap v656: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:21 smithi047 ceph-mon[35327]: pgmap v656: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:23 smithi038 ceph-mon[31227]: pgmap v657: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:23 smithi179 ceph-mon[35781]: pgmap v657: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:23.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:23 smithi047 ceph-mon[35327]: pgmap v657: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-09-19T05:51:25.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:25 smithi038 ceph-mon[31227]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T05:51:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:25 smithi179 ceph-mon[35781]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T05:51:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:25 smithi047 ceph-mon[35327]: pgmap v658: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-19T05:51:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:27 smithi038 ceph-mon[31227]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:27 smithi179 ceph-mon[35781]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:27.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:27 smithi047 ceph-mon[35327]: pgmap v659: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:29.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:29 smithi038 ceph-mon[31227]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:51:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:29 smithi179 ceph-mon[35781]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:51:29.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:29 smithi047 ceph-mon[35327]: pgmap v660: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:51:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:31 smithi038 ceph-mon[31227]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:31 smithi179 ceph-mon[35781]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:31.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:31 smithi047 ceph-mon[35327]: pgmap v661: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:33 smithi038 ceph-mon[31227]: pgmap v662: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:33 smithi179 ceph-mon[35781]: pgmap v662: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:33 smithi047 ceph-mon[35327]: pgmap v662: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:51:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:35 smithi038 ceph-mon[31227]: pgmap v663: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T05:51:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:35 smithi179 ceph-mon[35781]: pgmap v663: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T05:51:35.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:35 smithi047 ceph-mon[35327]: pgmap v663: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-19T05:51:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:37 smithi038 ceph-mon[31227]: pgmap v664: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:51:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:37 smithi179 ceph-mon[35781]: pgmap v664: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:51:37.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:37 smithi047 ceph-mon[35327]: pgmap v664: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:51:38.580 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:51:38.852 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:39.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 -- 172.21.15.38:0/2340745953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904100a80 msgr2=0x7f5904100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 --2- 172.21.15.38:0/2340745953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904100a80 0x7f5904100ee0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009930 tx=0x7f58f802a420 comp rx=0 tx=0).stop 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 -- 172.21.15.38:0/2340745953 shutdown_connections 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 --2- 172.21.15.38:0/2340745953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904101420 0x7f590410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 --2- 172.21.15.38:0/2340745953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904100a80 0x7f5904100ee0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 --2- 172.21.15.38:0/2340745953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f5904106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 -- 172.21.15.38:0/2340745953 >> 172.21.15.38:0/2340745953 conn(0x7f59040fc7a0 msgr2=0x7f59040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 -- 172.21.15.38:0/2340745953 shutdown_connections 2024-09-19T05:51:39.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.357+0000 7f590a9d9640 1 -- 172.21.15.38:0/2340745953 wait complete. 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 Processor -- start 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 -- start start 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904100a80 0x7f59040fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f59040fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5904110e80 con 0x7f5904101420 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5904110d00 con 0x7f5904100a80 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.358+0000 7f590a9d9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5904111000 con 0x7f5904106a80 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:39.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40248/0 (socket says 172.21.15.38:40248) 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 -- 172.21.15.38:0/3038282619 learned_addr learned my addr 172.21.15.38:0/3038282619 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f5903fff640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904100a80 0x7f59040fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f5908f76640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f59040fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 -- 172.21.15.38:0/3038282619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 msgr2=0x7f59040fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f59040fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 -- 172.21.15.38:0/3038282619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904100a80 msgr2=0x7f59040fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904100a80 0x7f59040fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 -- 172.21.15.38:0/3038282619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041a68e0 con 0x7f5904101420 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f5908f76640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f59040fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59037fe640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009230 tx=0x7f58f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f803f030 con 0x7f5904101420 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f8003ad0 con 0x7f5904101420 2024-09-19T05:51:39.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041a6b70 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8035070 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.359+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a7050 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.360+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904100ee0 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.361+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f58f8042030 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.362+0000 7f59017fa640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 0x7f58ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.362+0000 7f5903fff640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 0x7f58ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.362+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f58f80f0310 con 0x7f5904101420 2024-09-19T05:51:39.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.363+0000 7f5903fff640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 0x7f58ec0a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:39.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.366+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58f80bcc70 con 0x7f5904101420 2024-09-19T05:51:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.581+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f590410a020 con 0x7f5904101420 2024-09-19T05:51:39.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.582+0000 7f59017fa640 1 -- 172.21.15.38:0/3038282619 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f58f80dc020 con 0x7f5904101420 2024-09-19T05:51:39.587 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:51:39.587 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.584+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 msgr2=0x7f58ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.584+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 0x7f58ec0a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f400b390 comp rx=0 tx=0).stop 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.584+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 msgr2=0x7f59040fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.584+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009230 tx=0x7f58f8004270 comp rx=0 tx=0).stop 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 shutdown_connections 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f58ec0a3240 0x7f58ec0a5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5904106a80 0x7f59040fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5904101420 0x7f59040fd280 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 --2- 172.21.15.38:0/3038282619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5904100a80 0x7f59040fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 >> 172.21.15.38:0/3038282619 conn(0x7f59040fc7a0 msgr2=0x7f590406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 shutdown_connections 2024-09-19T05:51:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:39.585+0000 7f590a9d9640 1 -- 172.21.15.38:0/3038282619 wait complete. 2024-09-19T05:51:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:39 smithi038 ceph-mon[31227]: pgmap v665: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:39 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3038282619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:39 smithi179 ceph-mon[35781]: pgmap v665: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:39 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3038282619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:39 smithi047 ceph-mon[35327]: pgmap v665: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:39 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3038282619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:40.544 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 769a986b --release --await' 2024-09-19T05:51:40.544 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 769a986b --release --await 2024-09-19T05:51:40.814 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- 172.21.15.38:0/701708354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 msgr2=0x7f82f4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 --2- 172.21.15.38:0/701708354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4107b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009930 tx=0x7f82e402a420 comp rx=0 tx=0).stop 2024-09-19T05:51:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- 172.21.15.38:0/701708354 shutdown_connections 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 --2- 172.21.15.38:0/701708354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4108290 0x7f82f410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 --2- 172.21.15.38:0/701708354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4107b80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 --2- 172.21.15.38:0/701708354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4102d90 0x7f82f4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- 172.21.15.38:0/701708354 >> 172.21.15.38:0/701708354 conn(0x7f82f40fc7d0 msgr2=0x7f82f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- 172.21.15.38:0/701708354 shutdown_connections 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- 172.21.15.38:0/701708354 wait complete. 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 Processor -- start 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.341+0000 7f82fc431640 1 -- start start 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4102d90 0x7f82f4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4108290 0x7f82f419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112050 con 0x7f82f4103740 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111ed0 con 0x7f82f4102d90 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fc431640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41121d0 con 0x7f82f4108290 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40280/0 (socket says 172.21.15.38:40280) 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 -- 172.21.15.38:0/237658870 learned_addr learned my addr 172.21.15.38:0/237658870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4102d90 0x7f82f4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4108290 0x7f82f419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 -- 172.21.15.38:0/237658870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4108290 msgr2=0x7f82f419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4108290 0x7f82f419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 -- 172.21.15.38:0/237658870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4102d90 msgr2=0x7f82f4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4102d90 0x7f82f4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 -- 172.21.15.38:0/237658870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419d5c0 con 0x7f82f4103740 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82f99cc640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005940 tx=0x7f82e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.342+0000 7f82eb7fe640 1 -- 172.21.15.38:0/237658870 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e403f030 con 0x7f82f4103740 2024-09-19T05:51:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.343+0000 7f82eb7fe640 1 -- 172.21.15.38:0/237658870 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e4003ce0 con 0x7f82f4103740 2024-09-19T05:51:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.343+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419d850 con 0x7f82f4103740 2024-09-19T05:51:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.343+0000 7f82eb7fe640 1 -- 172.21.15.38:0/237658870 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4035070 con 0x7f82f4103740 2024-09-19T05:51:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.343+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a5190 con 0x7f82f4103740 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.345+0000 7f82eb7fe640 1 -- 172.21.15.38:0/237658870 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f82e4042030 con 0x7f82f4103740 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.345+0000 7f82fe80a740 1 Processor -- start 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.345+0000 7f82fe80a740 1 -- start start 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a04fd9f620 con 0x55a04fe3a980 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82eb7fe640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 0x7f82e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a04fd46e50 con 0x55a04fe3a060 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fe80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a04fd9b5e0 con 0x55a04fe45240 2024-09-19T05:51:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa1cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:55472/0 (socket says 172.21.15.38:55472) 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 -- 172.21.15.38:0/3415568260 learned_addr learned my addr 172.21.15.38:0/3415568260 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82eb7fe640 1 -- 172.21.15.38:0/237658870 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f82e40efe30 con 0x7f82f4103740 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 0x7f82e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82f99cc640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 -- 172.21.15.38:0/3415568260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 msgr2=0x55a04fe47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe47630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 -- 172.21.15.38:0/3415568260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 msgr2=0x55a04fe44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe44d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa9ce640 1 -- 172.21.15.38:0/3415568260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04fd37d40 con 0x55a04fe3a060 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82f99cc640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.346+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.347+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 0x7f82e00a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82ec010c40 tx=0x7f82ec00c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.347+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82ec012cf0 tx=0x7f82ec010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc2efae3b01af40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.347+0000 7f82e8dea640 1 -- 172.21.15.38:0/3415568260 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec022070 con 0x55a04fe3a060 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.347+0000 7f82e8dea640 1 -- 172.21.15.38:0/3415568260 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec009cf0 con 0x55a04fe3a060 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.347+0000 7f82e8dea640 1 -- 172.21.15.38:0/3415568260 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0122b0 con 0x55a04fe3a060 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- 172.21.15.38:0/3415568260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 msgr2=0x55a04fe3a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82ec012cf0 tx=0x7f82ec010740 comp rx=0 tx=0).stop 2024-09-19T05:51:41.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- 172.21.15.38:0/3415568260 shutdown_connections 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3415568260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe3a440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- 172.21.15.38:0/3415568260 >> 172.21.15.38:0/3415568260 conn(0x55a04fe365f0 msgr2=0x55a04fe369d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- 172.21.15.38:0/3415568260 shutdown_connections 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- 172.21.15.38:0/3415568260 wait complete. 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 -- start start 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.348+0000 7f82fe80a740 1 Processor -- start 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe42b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a04fe47e10 con 0x55a04fe3a980 2024-09-19T05:51:41.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a04fd27f10 con 0x55a04fe3a060 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a04fe47f90 con 0x55a04fe45240 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 10 client.0 _ll_register_callbacks cb 0x55a04fd4bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fa1cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40290/0 (socket says 172.21.15.38:40290) 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 -- 172.21.15.38:0/2812372624 learned_addr learned my addr 172.21.15.38:0/2812372624 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fa1cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:55488/0 (socket says 172.21.15.38:55488) 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe42b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 -- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 msgr2=0x55a04fe42b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe42b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 -- 172.21.15.38:0/2812372624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 msgr2=0x55a04fe420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 -- 172.21.15.38:0/2812372624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04faae580 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009320 tx=0x7f82e40bbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ce76cf967be204 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4049070 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f82e403b460 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.349+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a04fdacb90 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82e40044e0 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a04fb78960 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a04fdacb90 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82e40b87b0 con 0x55a04fe3a980 2024-09-19T05:51:41.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 10 client.15858 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 20 client.15858 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04fe54c00 0x55a04fe57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 4 client.15858 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.350+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04fda70a0 con 0x55a04fe54c00 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.351+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04fe54c00 0x55a04fe57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.351+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04fe54c00 0x55a04fe57060 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d124f1b3dc60610d server_cookie=279f24a4e3c1b2a3 in_seq=0 out_seq=0 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.351+0000 7f82ddffb640 10 client.15858 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82ddffb640 1 -- 172.21.15.38:0/2812372624 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a04fda70a0 con 0x55a04fe54c00 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82ddffb640 10 client.15858 handle_command_reply: tid=0 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 client.15858 shutdown 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 msgr2=0x55a04fe42620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009320 tx=0x7f82e40bbf90 comp rx=0 tx=0).stop 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 shutdown_connections 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04fe54c00 0x55a04fe57060 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe45240 0x55a04fe42b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82ddffb640 0 client.15858 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe42620 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2812372624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe3a060 0x55a04fe420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 >> 172.21.15.38:0/2812372624 conn(0x55a04fe365f0 msgr2=0x55a04fe35750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 shutdown_connections 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 1 -- 172.21.15.38:0/2812372624 wait complete. 2024-09-19T05:51:41.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.352+0000 7f82fe80a740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 Processor -- start 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 -- start start 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe5c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41aed10 con 0x55a04fe3a980 2024-09-19T05:51:41.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41b0850 con 0x55a04fe45240 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82fe80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41af3e0 con 0x55a04fe3a060 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.369+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40302/0 (socket says 172.21.15.38:40302) 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 -- 172.21.15.38:0/3008138803 learned_addr learned my addr 172.21.15.38:0/3008138803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe5c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 -- 172.21.15.38:0/3008138803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 msgr2=0x55a04fe409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 -- 172.21.15.38:0/3008138803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 msgr2=0x55a04fe5c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe5c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 -- 172.21.15.38:0/3008138803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04fe3dac0 con 0x55a04fe3a980 2024-09-19T05:51:41.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82fa9ce640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe5c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:51:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82f99cc640 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82e40bbe40 tx=0x7f82e40bbb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da5a50535559f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82dffff640 1 -- 172.21.15.38:0/3008138803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4049050 con 0x55a04fe3a980 2024-09-19T05:51:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82dffff640 1 -- 172.21.15.38:0/3008138803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e40bc140 con 0x55a04fe3a980 2024-09-19T05:51:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.370+0000 7f82dffff640 1 -- 172.21.15.38:0/3008138803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40b87b0 con 0x55a04fe3a980 2024-09-19T05:51:41.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- 172.21.15.38:0/3008138803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 msgr2=0x55a04fe3c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f82e40bbe40 tx=0x7f82e40bbb60 comp rx=0 tx=0).stop 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- 172.21.15.38:0/3008138803 shutdown_connections 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe5c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe3c1c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 --2- 172.21.15.38:0/3008138803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- 172.21.15.38:0/3008138803 >> 172.21.15.38:0/3008138803 conn(0x55a04fe365f0 msgr2=0x55a04fe3bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- 172.21.15.38:0/3008138803 shutdown_connections 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- 172.21.15.38:0/3008138803 wait complete. 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 -- start start 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 Processor -- start 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.371+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe4d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41b0180 con 0x55a04fe3a980 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41afab0 con 0x55a04fe45240 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41accf0 con 0x55a04fe3a060 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 10 client.0 _ll_register_callbacks cb 0x7f82e40b7e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fa1cd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe4d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fa9ce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fa1cd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe4d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.38:43738/0 (socket says 172.21.15.38:43738) 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40314/0 (socket says 172.21.15.38:40314) 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fa1cd640 1 -- 172.21.15.38:0/2937582008 learned_addr learned my addr 172.21.15.38:0/2937582008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 -- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 msgr2=0x55a04fe4d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe4d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 -- 172.21.15.38:0/2937582008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 msgr2=0x55a04fe4a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe4a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 -- 172.21.15.38:0/2937582008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04fe410d0 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82f99cc640 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55a04fe431d0 tx=0x7f82e40bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ea4b35d83823e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4049070 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f82e4031bd0 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.372+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e40bc140 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82e40c0440 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a04fe410d0 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e40bc140 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82e40c4870 con 0x55a04fe3a980 2024-09-19T05:51:41.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 10 client.15864 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 20 client.15864 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04febffd0 0x55a04fec23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 4 client.15864 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "769a986b", "release": true, "await": true}] 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "769a986b", "release": true, "await": true}) -- 0x55a04fd235a0 con 0x55a04febffd0 2024-09-19T05:51:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.373+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04febffd0 0x55a04fec23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.374+0000 7f82fa1cd640 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04febffd0 0x55a04fec23c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5132de7d808fe2aa server_cookie=931fcf354e3ca86c in_seq=0 out_seq=0 2024-09-19T05:51:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.374+0000 7f82ddffb640 10 client.15864 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82ddffb640 1 -- 172.21.15.38:0/2937582008 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a04fd235a0 con 0x55a04febffd0 2024-09-19T05:51:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82ddffb640 10 client.15864 handle_command_reply: tid=0 2024-09-19T05:51:41.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 client.15864 shutdown 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 msgr2=0x55a04fe4d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55a04fe431d0 tx=0x7f82e40bbbd0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 shutdown_connections 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a04febffd0 0x55a04fec23c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a04fe45240 0x55a04fe4a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82ddffb640 0 client.15864 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a04fe3a980 0x55a04fe4d720 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 --2- 172.21.15.38:0/2937582008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a04fe3a060 0x55a04fe4d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 >> 172.21.15.38:0/2937582008 conn(0x55a04fe365f0 msgr2=0x55a04fe420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 shutdown_connections 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 1 -- 172.21.15.38:0/2937582008 wait complete. 2024-09-19T05:51:41.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fe80a740 20 client.15864 trim_cache size 0 max 16384 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 36, 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "769a986b": { 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 36, 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:41.384 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.0, 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:41.385 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 msgr2=0x7f82e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 0x7f82e00a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82ec010c40 tx=0x7f82ec00c000 comp rx=0 tx=0).stop 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 msgr2=0x7f82f4198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.377+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005940 tx=0x7f82e40042f0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 shutdown_connections 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f82e00a3240 0x7f82e00a5700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4108290 0x7f82f419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82f4103740 0x7f82f4198b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 --2- 172.21.15.38:0/237658870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82f4102d90 0x7f82f4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 >> 172.21.15.38:0/237658870 conn(0x7f82f40fc7d0 msgr2=0x7f82f4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 shutdown_connections 2024-09-19T05:51:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:41.378+0000 7f82fc431640 1 -- 172.21.15.38:0/237658870 wait complete. 2024-09-19T05:51:41.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:41 smithi038 ceph-mon[31227]: pgmap v666: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:41 smithi179 ceph-mon[35781]: pgmap v666: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:41.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:41 smithi047 ceph-mon[35327]: pgmap v666: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:51:42.183 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '769a986b', total seconds elapsed: 33.6 2024-09-19T05:51:42.183 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T05:51:42.183 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:51:42.441 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:42.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 -- 172.21.15.38:0/34349266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8101de0 msgr2=0x7faef81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 --2- 172.21.15.38:0/34349266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8101de0 0x7faef81021c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faef4009900 tx=0x7faef402a410 comp rx=0 tx=0).stop 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 -- 172.21.15.38:0/34349266 shutdown_connections 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 --2- 172.21.15.38:0/34349266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8106f10 0x7faef810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 --2- 172.21.15.38:0/34349266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 0x7faef81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 --2- 172.21.15.38:0/34349266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8101de0 0x7faef81021c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 -- 172.21.15.38:0/34349266 >> 172.21.15.38:0/34349266 conn(0x7faef80fd8c0 msgr2=0x7faef80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 -- 172.21.15.38:0/34349266 shutdown_connections 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.986+0000 7faf00302640 1 -- 172.21.15.38:0/34349266 wait complete. 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 Processor -- start 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 -- start start 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8101de0 0x7faef8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:42.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 0x7faef8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faef8111f30 con 0x7faef8106f10 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faef8111db0 con 0x7faef8101de0 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faf00302640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faef81120b0 con 0x7faef8102790 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faefe89f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faefd89d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 0x7faef8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.987+0000 7faefe89f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40332/0 (socket says 172.21.15.38:40332) 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 -- 172.21.15.38:0/3108455906 learned_addr learned my addr 172.21.15.38:0/3108455906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe09e640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8101de0 0x7faef8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 -- 172.21.15.38:0/3108455906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 msgr2=0x7faef8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 0x7faef8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 -- 172.21.15.38:0/3108455906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8101de0 msgr2=0x7faef8198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:42.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8101de0 0x7faef8198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 -- 172.21.15.38:0/3108455906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef819d4c0 con 0x7faef8106f10 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faefe89f640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faef000eaa0 tx=0x7faef000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000cdb0 con 0x7faef8106f10 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0005020 con 0x7faef8106f10 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef819d7b0 con 0x7faef8106f10 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00116b0 con 0x7faef8106f10 2024-09-19T05:51:42.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.988+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a5090 con 0x7faef8106f10 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.989+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef81021c0 con 0x7faef8106f10 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.990+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faef0005560 con 0x7faef8106f10 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.991+0000 7faeef7fe640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 0x7faee40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.991+0000 7faefe09e640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 0x7faee40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.991+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7faef00cfd80 con 0x7faef8106f10 2024-09-19T05:51:42.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.992+0000 7faefe09e640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 0x7faee40a5780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faef4009230 tx=0x7faef40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:42.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:42.995+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faef0016030 con 0x7faef8106f10 2024-09-19T05:51:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.208+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef810a0f0 con 0x7faef8106f10 2024-09-19T05:51:43.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.209+0000 7faeef7fe640 1 -- 172.21.15.38:0/3108455906 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7faef009c090 con 0x7faef8106f10 2024-09-19T05:51:43.211 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:51:43.211 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:51:43.211 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 msgr2=0x7faee40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 0x7faee40a5780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faef4009230 tx=0x7faef40058d0 comp rx=0 tx=0).stop 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 msgr2=0x7faef819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faef000eaa0 tx=0x7faef000ef60 comp rx=0 tx=0).stop 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 shutdown_connections 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7faee40a32c0 0x7faee40a5780 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef8106f10 0x7faef819cde0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef8102790 0x7faef8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:43.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 --2- 172.21.15.38:0/3108455906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faef8101de0 0x7faef8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 >> 172.21.15.38:0/3108455906 conn(0x7faef80fd8c0 msgr2=0x7faef80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 shutdown_connections 2024-09-19T05:51:43.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:43.211+0000 7faf00302640 1 -- 172.21.15.38:0/3108455906 wait complete. 2024-09-19T05:51:43.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:43 smithi038 ceph-mon[31227]: pgmap v667: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T05:51:43.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:43 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3108455906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:43 smithi179 ceph-mon[35781]: pgmap v667: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T05:51:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:43 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3108455906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:43.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:43 smithi047 ceph-mon[35327]: pgmap v667: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-19T05:51:43.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:43 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3108455906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:51:44.166 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T05:51:44.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T05:51:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:51:44.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 -- 172.21.15.38:0/2650991188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4100950 msgr2=0x7f93c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 --2- 172.21.15.38:0/2650991188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4100950 0x7f93c4100d30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009900 tx=0x7f93b802a410 comp rx=0 tx=0).stop 2024-09-19T05:51:44.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 -- 172.21.15.38:0/2650991188 shutdown_connections 2024-09-19T05:51:44.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 --2- 172.21.15.38:0/2650991188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c410e500 0x7f93c41108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 --2- 172.21.15.38:0/2650991188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c4101300 0x7f93c410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 --2- 172.21.15.38:0/2650991188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4100950 0x7f93c4100d30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 -- 172.21.15.38:0/2650991188 >> 172.21.15.38:0/2650991188 conn(0x7f93c40fc7a0 msgr2=0x7f93c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 -- 172.21.15.38:0/2650991188 shutdown_connections 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.915+0000 7f93cad49640 1 -- 172.21.15.38:0/2650991188 wait complete. 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 Processor -- start 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 -- start start 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c4100950 0x7f93c419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c410e500 0x7f93c419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114200 con 0x7f93c4101300 2024-09-19T05:51:44.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114080 con 0x7f93c4100950 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93cad49640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114380 con 0x7f93c410e500 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40340/0 (socket says 172.21.15.38:40340) 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93c3fff640 1 -- 172.21.15.38:0/2553101822 learned_addr learned my addr 172.21.15.38:0/2553101822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93c92e6640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c410e500 0x7f93c419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.916+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c4100950 0x7f93c419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 -- 172.21.15.38:0/2553101822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c410e500 msgr2=0x7f93c419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c410e500 0x7f93c419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 -- 172.21.15.38:0/2553101822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c4100950 msgr2=0x7f93c419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c4100950 0x7f93c419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 -- 172.21.15.38:0/2553101822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c419f630 con 0x7f93c4101300 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d770 tx=0x7f93b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c1ffb640 1 -- 172.21.15.38:0/2553101822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0017070 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c1ffb640 1 -- 172.21.15.38:0/2553101822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b0005500 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93c1ffb640 1 -- 172.21.15.38:0/2553101822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004e00 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c419f920 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.917+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a7200 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.919+0000 7f93c1ffb640 1 -- 172.21.15.38:0/2553101822 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f93b0004fa0 con 0x7f93c4101300 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 Processor -- start 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 -- start start 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d2d760 0x556914d2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 0x556914d3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556914c932a0 con 0x556914d2e080 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556914c3ab10 con 0x556914d2d760 2024-09-19T05:51:44.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93cd122740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556914c8f260 con 0x556914d38940 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c1ffb640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 0x7f93b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40346/0 (socket says 172.21.15.38:40346) 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3436262152 learned_addr learned my addr 172.21.15.38:0/3436262152 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c92e6640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d2d760 0x556914d2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c3fff640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 0x556914d3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.920+0000 7f93c92e6640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 0x7f93b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c1ffb640 1 -- 172.21.15.38:0/2553101822 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f93b00cf700 con 0x7f93c4101300 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3436262152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 msgr2=0x556914d3ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 0x556914d3ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3436262152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d2d760 msgr2=0x556914d2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d2d760 0x556914d2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3436262152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556914a04b90 con 0x556914d2e080 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c3fff640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 0x556914d3ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009900 tx=0x7f93b80040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491280549705a77a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93c92e6640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 0x7f93b40a5700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93c406a0f0 tx=0x7f93bc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93aeffd640 1 -- 172.21.15.38:0/3436262152 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8035040 con 0x556914d2e080 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93aeffd640 1 -- 172.21.15.38:0/3436262152 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b8005520 con 0x556914d2e080 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.921+0000 7f93aeffd640 1 -- 172.21.15.38:0/3436262152 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8003850 con 0x556914d2e080 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- 172.21.15.38:0/3436262152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 msgr2=0x556914d38400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009900 tx=0x7f93b80040f0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- 172.21.15.38:0/3436262152 shutdown_connections 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d38940 0x556914d3ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2e080 0x556914d38400 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 --2- 172.21.15.38:0/3436262152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d2d760 0x556914d2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- 172.21.15.38:0/3436262152 >> 172.21.15.38:0/3436262152 conn(0x556914d29cf0 msgr2=0x556914d2a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- 172.21.15.38:0/3436262152 shutdown_connections 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- 172.21.15.38:0/3436262152 wait complete. 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 -- start start 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.922+0000 7f93cd122740 1 Processor -- start 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d35d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556914d3b510 con 0x556914d2d760 2024-09-19T05:51:44.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556914c1b6e0 con 0x556914d38940 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556914d3b690 con 0x556914d2e080 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93cd122740 10 client.0 _ll_register_callbacks cb 0x556914c3fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40362/0 (socket says 172.21.15.38:40362) 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 -- 172.21.15.38:0/1899655243 learned_addr learned my addr 172.21.15.38:0/1899655243 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c3fff640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d35d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c92e6640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 -- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 msgr2=0x556914d35d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d35d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 -- 172.21.15.38:0/1899655243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 msgr2=0x556914d36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 -- 172.21.15.38:0/1899655243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556914d449d0 con 0x556914d2d760 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93b80045d0 tx=0x7f93b8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c8c97acb25ef9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.923+0000 7f93a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8053060 con 0x556914d2d760 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f93b8004990 con 0x556914d2d760 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b8005520 con 0x556914d2d760 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93b805a050 con 0x556914d2d760 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:44.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556914d47340 con 0x556914d2d760 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b8005520 con 0x556914d2d760 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f93b80315b0 con 0x556914d2d760 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 10 client.15882 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 20 client.15882 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914d48690 0x556914d4aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 4 client.15882 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.924+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556914ca09d0 con 0x556914d48690 2024-09-19T05:51:44.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.925+0000 7f93c3fff640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914d48690 0x556914d4aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.925+0000 7f93c3fff640 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914d48690 0x556914d4aa80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb322075e31d2507 server_cookie=461aece08b00a8e7 in_seq=0 out_seq=0 2024-09-19T05:51:44.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.925+0000 7f93a3fff640 10 client.15882 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93a3fff640 1 -- 172.21.15.38:0/1899655243 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556914ca09d0 con 0x556914d48690 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93a3fff640 10 client.15882 handle_command_reply: tid=0 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 client.15882 shutdown 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 msgr2=0x556914d357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93b80045d0 tx=0x7f93b8009650 comp rx=0 tx=0).stop 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 shutdown_connections 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914d48690 0x556914d4aa80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d35d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93a3fff640 0 client.15882 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 --2- 172.21.15.38:0/1899655243 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d357e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 >> 172.21.15.38:0/1899655243 conn(0x556914d29cf0 msgr2=0x556914d2fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 shutdown_connections 2024-09-19T05:51:44.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 1 -- 172.21.15.38:0/1899655243 wait complete. 2024-09-19T05:51:44.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.926+0000 7f93cd122740 20 client.15882 trim_cache size 0 max 16384 2024-09-19T05:51:44.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.943+0000 7f93cd122740 1 Processor -- start 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.943+0000 7f93cd122740 1 -- start start 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.943+0000 7f93cd122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.943+0000 7f93cd122740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.943+0000 7f93cd122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d32f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93cd122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b0e90 con 0x556914d2d760 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93cd122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b29d0 con 0x556914d38940 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93cd122740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b1560 con 0x556914d2e080 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40364/0 (socket says 172.21.15.38:40364) 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3082258582 learned_addr learned my addr 172.21.15.38:0/3082258582 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:44.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c3fff640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c92e6640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d32f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3082258582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 msgr2=0x556914d32a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d32a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3082258582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 msgr2=0x556914d32f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d32f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 -- 172.21.15.38:0/3082258582 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556914d29b20 con 0x556914d2d760 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c3fff640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c92e6640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d32f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f93b8031550 tx=0x7f93b8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2867dc94d41f198 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93ae7fc640 1 -- 172.21.15.38:0/3082258582 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8053040 con 0x556914d2d760 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93ae7fc640 1 -- 172.21.15.38:0/3082258582 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b80315b0 con 0x556914d2d760 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.944+0000 7f93ae7fc640 1 -- 172.21.15.38:0/3082258582 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8035050 con 0x556914d2d760 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- 172.21.15.38:0/3082258582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 msgr2=0x556914d37750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f93b8031550 tx=0x7f93b8005780 comp rx=0 tx=0).stop 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- 172.21.15.38:0/3082258582 shutdown_connections 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914d32f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 --2- 172.21.15.38:0/3082258582 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d37750 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- 172.21.15.38:0/3082258582 >> 172.21.15.38:0/3082258582 conn(0x556914d29cf0 msgr2=0x556914d37040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- 172.21.15.38:0/3082258582 shutdown_connections 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- 172.21.15.38:0/3082258582 wait complete. 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 -- start start 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.945+0000 7f93cd122740 1 Processor -- start 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914db1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b2300 con 0x556914d2d760 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b1c30 con 0x556914d38940 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41aee70 con 0x556914d2e080 2024-09-19T05:51:44.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93cd122740 10 client.0 _ll_register_callbacks cb 0x7f93b80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:51:44.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40370/0 (socket says 172.21.15.38:40370) 2024-09-19T05:51:44.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 -- 172.21.15.38:0/2853754948 learned_addr learned my addr 172.21.15.38:0/2853754948 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:51:44.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c92e6640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914db1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 -- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 msgr2=0x556914d4c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d4c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 -- 172.21.15.38:0/2853754948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 msgr2=0x556914db1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914db1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c8ae5640 1 -- 172.21.15.38:0/2853754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556914d4c970 con 0x556914d2d760 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.946+0000 7f93c92e6640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914db1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93c8ae5640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f93b80319b0 tx=0x7f93b8032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa4af52b953ed0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:51:44.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8063020 con 0x556914d2d760 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f93b8031760 con 0x556914d2d760 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b80315b0 con 0x556914d2d760 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93b806b050 con 0x556914d2d760 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556914db3530 con 0x556914d2d760 2024-09-19T05:51:44.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b80315b0 con 0x556914d2d760 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f93b805fb10 con 0x556914d2d760 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 10 client.15888 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 20 client.15888 populate_metadata read hostname 'smithi038' 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914db3c50 0x556914db6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 4 client.15888 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T05:51:44.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.947+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x556914c9ad20 con 0x556914db3c50 2024-09-19T05:51:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.948+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914db3c50 0x556914db6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:51:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.948+0000 7f93c3fff640 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914db3c50 0x556914db6070 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5231f6d8d3c3c0a3 server_cookie=7edd44cee62d9b36 in_seq=0 out_seq=0 2024-09-19T05:51:44.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.948+0000 7f93a3fff640 10 client.15888 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:44.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93a3fff640 1 -- 172.21.15.38:0/2853754948 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556914c9ad20 con 0x556914db3c50 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93a3fff640 10 client.15888 handle_command_reply: tid=0 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 client.15888 shutdown 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 msgr2=0x556914d4bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f93b80319b0 tx=0x7f93b8032f00 comp rx=0 tx=0).stop 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 shutdown_connections 2024-09-19T05:51:44.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x556914db3c50 0x556914db6070 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556914d38940 0x556914db1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556914d2e080 0x556914d4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93a3fff640 0 client.15888 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 --2- 172.21.15.38:0/2853754948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556914d2d760 0x556914d4bba0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 >> 172.21.15.38:0/2853754948 conn(0x556914d29cf0 msgr2=0x556914d37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 shutdown_connections 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 1 -- 172.21.15.38:0/2853754948 wait complete. 2024-09-19T05:51:44.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.959+0000 7f93cd122740 20 client.15888 trim_cache size 0 max 16384 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 38, 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "930417ee": { 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 38, 2024-09-19T05:51:44.966 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.0, 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:51:44.967 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:51:44.968 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 msgr2=0x7f93b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 0x7f93b40a5700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93c406a0f0 tx=0x7f93bc009600 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 msgr2=0x7f93c419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d770 tx=0x7f93b000dc30 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 shutdown_connections 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f93b40a3240 0x7f93b40a5700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93c410e500 0x7f93c419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93c4101300 0x7f93c419abc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 --2- 172.21.15.38:0/2553101822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93c4100950 0x7f93c419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:51:44.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 >> 172.21.15.38:0/2553101822 conn(0x7f93c40fc7a0 msgr2=0x7f93c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:51:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 shutdown_connections 2024-09-19T05:51:44.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:51:44.960+0000 7f93cad49640 1 -- 172.21.15.38:0/2553101822 wait complete. 2024-09-19T05:51:45.705 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:45 smithi038 ceph-mon[31227]: pgmap v668: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-19T05:51:45.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '930417ee', quiesce took 3.6 seconds. Will release after: 26.4 2024-09-19T05:51:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:45 smithi179 ceph-mon[35781]: pgmap v668: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-19T05:51:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:45 smithi047 ceph-mon[35327]: pgmap v668: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-19T05:51:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:47 smithi038 ceph-mon[31227]: pgmap v669: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T05:51:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:47 smithi179 ceph-mon[35781]: pgmap v669: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T05:51:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:47 smithi047 ceph-mon[35327]: pgmap v669: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-19T05:51:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:49 smithi038 ceph-mon[31227]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:51:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:49 smithi179 ceph-mon[35781]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:51:49.983 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:49 smithi047 ceph-mon[35327]: pgmap v670: 129 pgs: 129 active+clean; 4.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-19T05:51:51.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:51 smithi038 ceph-mon[31227]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:51 smithi179 ceph-mon[35781]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:51 smithi047 ceph-mon[35327]: pgmap v671: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:53.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:53 smithi038 ceph-mon[31227]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:53 smithi179 ceph-mon[35781]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:53.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:53 smithi047 ceph-mon[35327]: pgmap v672: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:51:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:55 smithi179 ceph-mon[35781]: pgmap v673: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:51:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:55 smithi047 ceph-mon[35327]: pgmap v673: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:51:56.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:55 smithi038 ceph-mon[31227]: pgmap v673: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:51:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:57 smithi179 ceph-mon[35781]: pgmap v674: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:51:57.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:57 smithi047 ceph-mon[35327]: pgmap v674: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:51:58.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:57 smithi038 ceph-mon[31227]: pgmap v674: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T05:51:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:51:59 smithi179 ceph-mon[35781]: pgmap v675: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T05:51:59.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:51:59 smithi047 ceph-mon[35327]: pgmap v675: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T05:52:00.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:51:59 smithi038 ceph-mon[31227]: pgmap v675: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T05:52:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:01 smithi179 ceph-mon[35781]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:52:01.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:01 smithi047 ceph-mon[35327]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:52:02.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:01 smithi038 ceph-mon[31227]: pgmap v676: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:52:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:03 smithi179 ceph-mon[35781]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:03 smithi047 ceph-mon[35327]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:04.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:03 smithi038 ceph-mon[31227]: pgmap v677: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:04.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:04 smithi179 ceph-mon[35781]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:04.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:04 smithi047 ceph-mon[35327]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:05.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:04 smithi038 ceph-mon[31227]: pgmap v678: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:52:07.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:06 smithi038 ceph-mon[31227]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:07.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:06 smithi179 ceph-mon[35781]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:07.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:06 smithi047 ceph-mon[35327]: pgmap v679: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:09.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:08 smithi179 ceph-mon[35781]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:52:09.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:08 smithi047 ceph-mon[35327]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:52:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:08 smithi038 ceph-mon[31227]: pgmap v680: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:52:11.196 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:10 smithi038 ceph-mon[31227]: pgmap v681: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:11.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:10 smithi179 ceph-mon[35781]: pgmap v681: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:11.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:10 smithi047 ceph-mon[35327]: pgmap v681: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:12.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:52:12.407 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:52:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- 172.21.15.38:0/1139725296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4102d90 msgr2=0x7fb9d4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 --2- 172.21.15.38:0/1139725296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4102d90 0x7fb9d4103170 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009930 tx=0x7fb9c402a420 comp rx=0 tx=0).stop 2024-09-19T05:52:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- 172.21.15.38:0/1139725296 shutdown_connections 2024-09-19T05:52:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 --2- 172.21.15.38:0/1139725296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4108290 0x7fb9d410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 --2- 172.21.15.38:0/1139725296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4103740 0x7fb9d4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 --2- 172.21.15.38:0/1139725296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4102d90 0x7fb9d4103170 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- 172.21.15.38:0/1139725296 >> 172.21.15.38:0/1139725296 conn(0x7fb9d40fc7d0 msgr2=0x7fb9d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- 172.21.15.38:0/1139725296 shutdown_connections 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- 172.21.15.38:0/1139725296 wait complete. 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 Processor -- start 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.939+0000 7fb9da340640 1 -- start start 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4102d90 0x7fb9d4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4103740 0x7fb9d4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4112090 con 0x7fb9d4108290 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4111f10 con 0x7fb9d4102d90 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9da340640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4112210 con 0x7fb9d4103740 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4102d90 0x7fb9d4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42356/0 (socket says 172.21.15.38:42356) 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 -- 172.21.15.38:0/3741569979 learned_addr learned my addr 172.21.15.38:0/3741569979 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d37fe640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4103740 0x7fb9d4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 -- 172.21.15.38:0/3741569979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4103740 msgr2=0x7fb9d4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4103740 0x7fb9d4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 -- 172.21.15.38:0/3741569979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4102d90 msgr2=0x7fb9d4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4102d90 0x7fb9d4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.940+0000 7fb9d88dd640 1 -- 172.21.15.38:0/3741569979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d419d500 con 0x7fb9d4108290 2024-09-19T05:52:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9d88dd640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00cad0 tx=0x7fb9cc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00ad60 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9cc005020 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d419d7f0 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00f6b0 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.941+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d41a50d0 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.942+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d406a7f0 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.943+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb9cc005560 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.944+0000 7fb9d17fa640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 0x7fb9bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.944+0000 7fb9d3fff640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 0x7fb9bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.944+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb9cc0cffb0 con 0x7fb9d4108290 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.944+0000 7fb9d3fff640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 0x7fb9bc0a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009900 tx=0x7fb9c40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:12.947+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9cc014030 con 0x7fb9d4108290 2024-09-19T05:52:13.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:12 smithi038 ceph-mon[31227]: pgmap v682: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:52:13.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.161+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9d4103170 con 0x7fb9d4108290 2024-09-19T05:52:13.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.162+0000 7fb9d17fa640 1 -- 172.21.15.38:0/3741569979 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fb9d4103170 con 0x7fb9d4108290 2024-09-19T05:52:13.165 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:52:13.165 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:52:13.165 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:52:13.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 msgr2=0x7fb9bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:13.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 0x7fb9bc0a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009900 tx=0x7fb9c40058d0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 msgr2=0x7fb9d419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00cad0 tx=0x7fb9cc00ced0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 shutdown_connections 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb9bc0a3240 0x7fb9bc0a5700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9d4108290 0x7fb9d419ce20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9d4103740 0x7fb9d4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 --2- 172.21.15.38:0/3741569979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9d4102d90 0x7fb9d4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 >> 172.21.15.38:0/3741569979 conn(0x7fb9d40fc7d0 msgr2=0x7fb9d4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 shutdown_connections 2024-09-19T05:52:13.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:13.164+0000 7fb9da340640 1 -- 172.21.15.38:0/3741569979 wait complete. 2024-09-19T05:52:13.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:12 smithi179 ceph-mon[35781]: pgmap v682: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:52:13.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:12 smithi047 ceph-mon[35327]: pgmap v682: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:52:13.893 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 930417ee --release --await' 2024-09-19T05:52:13.894 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 930417ee --release --await 2024-09-19T05:52:14.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:13 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3741569979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:13 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.162 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:52:14.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:13 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3741569979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:13 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:13 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3741569979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:13 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:52:14.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 -- 172.21.15.38:0/278417207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 msgr2=0x7fcd34106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/278417207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd34106e60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009930 tx=0x7fcd3002a420 comp rx=0 tx=0).stop 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 -- 172.21.15.38:0/278417207 shutdown_connections 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/278417207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 0x7fcd3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/278417207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 0x7fcd34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/278417207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd34106e60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 -- 172.21.15.38:0/278417207 >> 172.21.15.38:0/278417207 conn(0x7fcd340fc7a0 msgr2=0x7fcd340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 -- 172.21.15.38:0/278417207 shutdown_connections 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.667+0000 7fcd3b55e640 1 -- 172.21.15.38:0/278417207 wait complete. 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 Processor -- start 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 -- start start 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 0x7fcd3419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 0x7fcd3419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34110e40 con 0x7fcd34106a80 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34110cc0 con 0x7fcd34101420 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd3b55e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34110fc0 con 0x7fcd34100a80 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42370/0 (socket says 172.21.15.38:42370) 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd39afb640 1 -- 172.21.15.38:0/1383561307 learned_addr learned my addr 172.21.15.38:0/1383561307 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.668+0000 7fcd38af9640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 0x7fcd3419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd392fa640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 0x7fcd3419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 -- 172.21.15.38:0/1383561307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 msgr2=0x7fcd3419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 0x7fcd3419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 -- 172.21.15.38:0/1383561307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 msgr2=0x7fcd3419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 0x7fcd3419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 -- 172.21.15.38:0/1383561307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3419f600 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd39afb640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00eb10 tx=0x7fcd2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd267fc640 1 -- 172.21.15.38:0/1383561307 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c004b20 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd267fc640 1 -- 172.21.15.38:0/1383561307 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c004cc0 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3419f8f0 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd267fc640 1 -- 172.21.15.38:0/1383561307 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c010760 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.669+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a71d0 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.671+0000 7fcd267fc640 1 -- 172.21.15.38:0/1383561307 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcd2c005020 con 0x7fcd34106a80 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 Processor -- start 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 -- start start 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a8d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd267fc640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 0x7fcd200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 0x55a4f7a97d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a98240 0x55a4f7a9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a4f79f2620 con 0x55a4f7a8d980 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a4f7999e50 con 0x55a4f7a8d060 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd3d937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a4f79ee5e0 con 0x55a4f7a98240 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd392fa640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 0x7fcd200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd38af9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a8d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.672+0000 7fcd267fc640 1 -- 172.21.15.38:0/1383561307 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fcd2c0d01c0 con 0x7fcd34106a80 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a8d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.38:45814/0 (socket says 172.21.15.38:45814) 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 -- 172.21.15.38:0/3532492 learned_addr learned my addr 172.21.15.38:0/3532492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 0x55a4f7a97d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a98240 0x55a4f7a9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 -- 172.21.15.38:0/3532492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a98240 msgr2=0x55a4f7a9a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a98240 0x55a4f7a9a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 -- 172.21.15.38:0/3532492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 msgr2=0x55a4f7a8d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a8d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 -- 172.21.15.38:0/3532492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f798ad40 con 0x55a4f7a8d980 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd392fa640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 0x7fcd200a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 0x55a4f7a97d00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcd280127c0 tx=0x7fcd2800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b0a67e2c43e56d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd1f7fe640 1 -- 172.21.15.38:0/3532492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280096a0 con 0x55a4f7a8d980 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd1f7fe640 1 -- 172.21.15.38:0/3532492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28009d20 con 0x55a4f7a8d980 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.673+0000 7fcd1f7fe640 1 -- 172.21.15.38:0/3532492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28010b20 con 0x55a4f7a8d980 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 -- 172.21.15.38:0/3532492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 msgr2=0x55a4f7a97d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 0x55a4f7a97d00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcd280127c0 tx=0x7fcd2800b390 comp rx=0 tx=0).stop 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 -- 172.21.15.38:0/3532492 shutdown_connections 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a98240 0x55a4f7a9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d980 0x55a4f7a97d00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3532492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a8d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 -- 172.21.15.38:0/3532492 >> 172.21.15.38:0/3532492 conn(0x55a4f7a895f0 msgr2=0x55a4f7a899d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 -- 172.21.15.38:0/3532492 shutdown_connections 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.674+0000 7fcd3d937740 1 -- 172.21.15.38:0/3532492 wait complete. 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 -- start start 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 Processor -- start 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a98240 0x55a4f7a95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55a4f7a9ae10 con 0x55a4f7a8d060 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a4f797af10 con 0x55a4f7a98240 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a4f7a9af90 con 0x55a4f7a8d980 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd3d937740 10 client.0 _ll_register_callbacks cb 0x55a4f799edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd392fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd392fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42390/0 (socket says 172.21.15.38:42390) 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd392fa640 1 -- 172.21.15.38:0/2528311679 learned_addr learned my addr 172.21.15.38:0/2528311679 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.675+0000 7fcd38af9640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd39afb640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a98240 0x55a4f7a95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 -- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 msgr2=0x55a4f7a95730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a95730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 -- 172.21.15.38:0/2528311679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a98240 msgr2=0x55a4f7a95c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a98240 0x55a4f7a95c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 -- 172.21.15.38:0/2528311679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f7701580 con 0x55a4f7a8d060 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd392fa640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009930 tx=0x7fcd30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a96797bafe3a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3004d020 con 0x55a4f7a8d060 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fcd300037f0 con 0x55a4f7a8d060 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd28009d20 con 0x55a4f7a8d060 2024-09-19T05:52:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd30058050 con 0x55a4f7a8d060 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd3d937740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4f77cb960 con 0x55a4f7a8d060 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.676+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd28009d20 con 0x55a4f7a8d060 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fcd30049a90 con 0x55a4f7a8d060 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 10 client.15906 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 20 client.15906 populate_metadata read hostname 'smithi038' 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7aa7e70 0x55a4f7aaa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 4 client.15906 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f79ffb90 con 0x55a4f7aa7e70 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd38af9640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7aa7e70 0x55a4f7aaa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd38af9640 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7aa7e70 0x55a4f7aaa2e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd654aa91480278e server_cookie=2e3f08071f5499c2 in_seq=0 out_seq=0 2024-09-19T05:52:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.677+0000 7fcd1cff9640 10 client.15906 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd1cff9640 1 -- 172.21.15.38:0/2528311679 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4f79ffb90 con 0x55a4f7aa7e70 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd1cff9640 10 client.15906 handle_command_reply: tid=0 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 client.15906 shutdown 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 msgr2=0x55a4f7a951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009930 tx=0x7fcd30009260 comp rx=0 tx=0).stop 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 shutdown_connections 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7aa7e70 0x55a4f7aaa2e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a98240 0x55a4f7a95c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 --2- 172.21.15.38:0/2528311679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a8d060 0x55a4f7a951f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 >> 172.21.15.38:0/2528311679 conn(0x55a4f7a895f0 msgr2=0x55a4f7a8f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.678+0000 7fcd1cff9640 0 client.15906 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:52:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.679+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 shutdown_connections 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.679+0000 7fcd3d937740 1 -- 172.21.15.38:0/2528311679 wait complete. 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.679+0000 7fcd3d937740 20 client.15906 trim_cache size 0 max 16384 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 Processor -- start 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 -- start start 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7aafc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341b0f70 con 0x55a4f7a98240 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341b2ab0 con 0x55a4f7a8d060 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd3d937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341b1640 con 0x55a4f7a8d980 2024-09-19T05:52:14.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42402/0 (socket says 172.21.15.38:42402) 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 -- 172.21.15.38:0/3602003461 learned_addr learned my addr 172.21.15.38:0/3602003461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd392fa640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7aafc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 -- 172.21.15.38:0/3602003461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 msgr2=0x55a4f7aafc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7aafc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 -- 172.21.15.38:0/3602003461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 msgr2=0x55a4f7a958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd39afb640 1 -- 172.21.15.38:0/3602003461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f7a8d7b0 con 0x55a4f7a98240 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.696+0000 7fcd392fa640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0cd900 tx=0x7fcd2c096f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7e8a153b3f07cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd1effd640 1 -- 172.21.15.38:0/3602003461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b4070 con 0x55a4f7a98240 2024-09-19T05:52:14.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd1effd640 1 -- 172.21.15.38:0/3602003461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c09cb20 con 0x55a4f7a98240 2024-09-19T05:52:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd1effd640 1 -- 172.21.15.38:0/3602003461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b5270 con 0x55a4f7a98240 2024-09-19T05:52:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 -- 172.21.15.38:0/3602003461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 msgr2=0x55a4f7ab0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0cd900 tx=0x7fcd2c096f90 comp rx=0 tx=0).stop 2024-09-19T05:52:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 -- 172.21.15.38:0/3602003461 shutdown_connections 2024-09-19T05:52:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7ab0140 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7aafc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3602003461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 -- 172.21.15.38:0/3602003461 >> 172.21.15.38:0/3602003461 conn(0x55a4f7a895f0 msgr2=0x55a4f7a951b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.697+0000 7fcd3d937740 1 -- 172.21.15.38:0/3602003461 shutdown_connections 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 -- 172.21.15.38:0/3602003461 wait complete. 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 -- start start 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 Processor -- start 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a9c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341b23e0 con 0x55a4f7a98240 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341b1d10 con 0x55a4f7a8d060 2024-09-19T05:52:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341aef50 con 0x55a4f7a8d980 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.698+0000 7fcd3d937740 10 client.0 _ll_register_callbacks cb 0x7fcd30031630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42414/0 (socket says 172.21.15.38:42414) 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 -- 172.21.15.38:0/3873855028 learned_addr learned my addr 172.21.15.38:0/3873855028 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd38af9640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a9c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd392fa640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a9c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 -- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 msgr2=0x55a4f7a9c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a9c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 -- 172.21.15.38:0/3873855028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 msgr2=0x55a4f7a9c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a9c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 -- 172.21.15.38:0/3873855028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f7b11dd0 con 0x55a4f7a98240 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd39afb640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c098010 tx=0x7fcd2c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66731d888565144a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:52:14.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c07f030 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fcd2c0b5270 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c09cb20 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c0b4050 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd3d937740 10 client.15912 fetch_fsmap learned FSMap version 28 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.699+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4f7b130b0 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c09cb20 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fcd2c005310 con 0x55a4f7a98240 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 10 client.15912 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 20 client.15912 populate_metadata read hostname 'smithi038' 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7b13760 0x55a4f7b15b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:52:14.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 4 client.15912 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "930417ee", "release": true, "await": true}] 2024-09-19T05:52:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "930417ee", "release": true, "await": true}) -- 0x55a4f79fa0a0 con 0x55a4f7b13760 2024-09-19T05:52:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.700+0000 7fcd392fa640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7b13760 0x55a4f7b15b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:52:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.701+0000 7fcd392fa640 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7b13760 0x55a4f7b15b50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6c36a44d3e56d5 server_cookie=6d7cd15b85380f80 in_seq=0 out_seq=0 2024-09-19T05:52:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.701+0000 7fcd1cff9640 10 client.15912 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:52:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd1cff9640 1 -- 172.21.15.38:0/3873855028 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a4f79fa0a0 con 0x55a4f7b13760 2024-09-19T05:52:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd1cff9640 10 client.15912 handle_command_reply: tid=0 2024-09-19T05:52:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 client.15912 shutdown 2024-09-19T05:52:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 msgr2=0x55a4f7a9cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c098010 tx=0x7fcd2c005c70 comp rx=0 tx=0).stop 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 shutdown_connections 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55a4f7b13760 0x55a4f7b15b50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55a4f7a98240 0x55a4f7a9cb80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4f7a8d980 0x55a4f7a9c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 --2- 172.21.15.38:0/3873855028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a4f7a8d060 0x55a4f7a9c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd1cff9640 0 client.15912 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 >> 172.21.15.38:0/3873855028 conn(0x55a4f7a895f0 msgr2=0x55a4f7a97270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 shutdown_connections 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 1 -- 172.21.15.38:0/3873855028 wait complete. 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.704+0000 7fcd3d937740 20 client.15912 trim_cache size 0 max 16384 2024-09-19T05:52:14.721 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 40, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "930417ee": { 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 40, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:52:14.722 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.0, 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:52:14.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:52:14.724 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:52:14.724 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:52:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 msgr2=0x7fcd200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 0x7fcd200a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd300058d0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 msgr2=0x7fcd3419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00eb10 tx=0x7fcd2c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 shutdown_connections 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fcd200a3240 0x7fcd200a5700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd34106a80 0x7fcd3419ef20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd34101420 0x7fcd3419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 --2- 172.21.15.38:0/1383561307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34100a80 0x7fcd3419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 >> 172.21.15.38:0/1383561307 conn(0x7fcd340fc7a0 msgr2=0x7fcd3406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 shutdown_connections 2024-09-19T05:52:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:52:14.705+0000 7fcd3b55e640 1 -- 172.21.15.38:0/1383561307 wait complete. 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: pgmap v683: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:52:15.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:14 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.327 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: pgmap v683: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:52:15.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:52:15.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:52:15.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:14 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.437 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '930417ee', total seconds elapsed: 33.3 2024-09-19T05:52:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: pgmap v683: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:52:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:52:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:52:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:14 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:52:17.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:16 smithi047 ceph-mon[35327]: pgmap v684: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:16 smithi038 ceph-mon[31227]: pgmap v684: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:16 smithi179 ceph-mon[35781]: pgmap v684: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:52:19.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:18 smithi047 ceph-mon[35327]: pgmap v685: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:18 smithi038 ceph-mon[31227]: pgmap v685: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:18 smithi179 ceph-mon[35781]: pgmap v685: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:21.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:20 smithi047 ceph-mon[35327]: pgmap v686: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:21.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:20 smithi038 ceph-mon[31227]: pgmap v686: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:20 smithi179 ceph-mon[35781]: pgmap v686: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:23.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:22 smithi038 ceph-mon[31227]: pgmap v687: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:23.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:22 smithi179 ceph-mon[35781]: pgmap v687: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:23.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:22 smithi047 ceph-mon[35327]: pgmap v687: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:52:25.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:24 smithi038 ceph-mon[31227]: pgmap v688: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-09-19T05:52:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:24 smithi179 ceph-mon[35781]: pgmap v688: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-09-19T05:52:25.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:24 smithi047 ceph-mon[35327]: pgmap v688: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-09-19T05:52:27.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:26 smithi038 ceph-mon[31227]: pgmap v689: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-19T05:52:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:26 smithi179 ceph-mon[35781]: pgmap v689: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-19T05:52:27.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:26 smithi047 ceph-mon[35327]: pgmap v689: 129 pgs: 129 active+clean; 4.7 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2024-09-19T05:52:29.249 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:28 smithi179 ceph-mon[35781]: pgmap v690: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T05:52:29.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:28 smithi038 ceph-mon[31227]: pgmap v690: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T05:52:29.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:28 smithi047 ceph-mon[35327]: pgmap v690: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-19T05:52:31.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:30 smithi038 ceph-mon[31227]: pgmap v691: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:30 smithi179 ceph-mon[35781]: pgmap v691: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:31.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:30 smithi047 ceph-mon[35327]: pgmap v691: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:33.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:32 smithi038 ceph-mon[31227]: pgmap v692: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:32 smithi179 ceph-mon[35781]: pgmap v692: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:33.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:32 smithi047 ceph-mon[35327]: pgmap v692: 129 pgs: 129 active+clean; 4.8 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:52:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:35 smithi038 ceph-mon[31227]: pgmap v693: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:35 smithi179 ceph-mon[35781]: pgmap v693: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:35.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:35 smithi047 ceph-mon[35327]: pgmap v693: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:37 smithi038 ceph-mon[31227]: pgmap v694: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:52:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:37 smithi179 ceph-mon[35781]: pgmap v694: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:52:37.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:37 smithi047 ceph-mon[35327]: pgmap v694: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:52:39.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:39 smithi038 ceph-mon[31227]: pgmap v695: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:52:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:39 smithi179 ceph-mon[35781]: pgmap v695: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:52:39.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:39 smithi047 ceph-mon[35327]: pgmap v695: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:52:41.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:41 smithi038 ceph-mon[31227]: pgmap v696: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:41.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:41 smithi179 ceph-mon[35781]: pgmap v696: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:41.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:41 smithi047 ceph-mon[35327]: pgmap v696: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:42.905 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:313/600: open fd = 3 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:314/600: open fd = 3 2024-09-19T05:52:42.906 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:315/600: open fd = 3 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:316/600: open fd = 3 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:317/600: open fd = 3 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.907 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:318/600: open fd = 3 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:319/600: open fd = 3 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:320/600: open fd = 3 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:321/600: open fd = 3 2024-09-19T05:52:42.908 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:322/600: open fd = 3 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:323/600: open fd = 3 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:324/600: open fd = 3 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.909 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:325/600: open fd = 3 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:326/600: open fd = 3 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:327/600: open fd = 3 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.910 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:328/600: open fd = 3 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:329/600: open fd = 3 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:330/600: open fd = 3 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.911 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:331/600: open fd = 3 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:332/600: open fd = 3 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:333/600: open fd = 3 2024-09-19T05:52:42.912 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:334/600: open fd = 3 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:335/600: open fd = 3 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:336/600: open fd = 3 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.913 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:337/600: open fd = 3 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:338/600: open fd = 3 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:339/600: open fd = 3 2024-09-19T05:52:42.914 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:340/600: open fd = 3 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:341/600: open fd = 3 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:342/600: open fd = 3 2024-09-19T05:52:42.915 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:343/600: open fd = 3 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:344/600: open fd = 3 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:345/600: open fd = 3 2024-09-19T05:52:42.916 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:346/600: open fd = 3 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:347/600: open fd = 3 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.917 INFO:tasks.workunit.client.0.smithi038.stdout:348/600: open fd = 3 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:349/600: open fd = 3 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:350/600: open fd = 3 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.918 INFO:tasks.workunit.client.0.smithi038.stdout:351/600: open fd = 3 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:352/600: open fd = 3 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:353/600: open fd = 3 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.919 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:354/600: open fd = 3 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:355/600: open fd = 3 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:356/600: open fd = 3 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.920 INFO:tasks.workunit.client.0.smithi038.stdout:357/600: open fd = 3 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:358/600: open fd = 3 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:359/600: open fd = 3 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.921 INFO:tasks.workunit.client.0.smithi038.stdout:360/600: open fd = 3 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:361/600: open fd = 3 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:362/600: open fd = 3 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.922 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:363/600: open fd = 3 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:364/600: open fd = 3 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:365/600: open fd = 3 2024-09-19T05:52:42.923 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:366/600: open fd = 3 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:367/600: open fd = 3 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.924 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:368/600: open fd = 3 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:369/600: open fd = 3 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.925 INFO:tasks.workunit.client.0.smithi038.stdout:370/600: open fd = 3 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:371/600: open fd = 3 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:372/600: open fd = 3 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.926 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:373/600: open fd = 3 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:374/600: open fd = 3 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:52:42.927 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:52:43.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:43 smithi038 ceph-mon[31227]: pgmap v697: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:43.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:43 smithi179 ceph-mon[35781]: pgmap v697: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:43.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:43 smithi047 ceph-mon[35327]: pgmap v697: 129 pgs: 129 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:45.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:45 smithi038 ceph-mon[31227]: pgmap v698: 129 pgs: 129 active+clean; 5.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:45 smithi179 ceph-mon[35781]: pgmap v698: 129 pgs: 129 active+clean; 5.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:45 smithi047 ceph-mon[35327]: pgmap v698: 129 pgs: 129 active+clean; 5.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:52:47.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:47 smithi038 ceph-mon[31227]: pgmap v699: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:52:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:47 smithi179 ceph-mon[35781]: pgmap v699: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:52:47.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:47 smithi047 ceph-mon[35327]: pgmap v699: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:52:49.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:49 smithi038 ceph-mon[31227]: pgmap v700: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:49 smithi179 ceph-mon[35781]: pgmap v700: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:49 smithi047 ceph-mon[35327]: pgmap v700: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:51.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:51 smithi038 ceph-mon[31227]: pgmap v701: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:51 smithi179 ceph-mon[35781]: pgmap v701: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:51.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:51 smithi047 ceph-mon[35327]: pgmap v701: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:52:53.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:53 smithi038 ceph-mon[31227]: pgmap v702: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:53 smithi179 ceph-mon[35781]: pgmap v702: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:53.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:53 smithi047 ceph-mon[35327]: pgmap v702: 129 pgs: 129 active+clean; 5.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:55.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:55 smithi038 ceph-mon[31227]: pgmap v703: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:55.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:55 smithi179 ceph-mon[35781]: pgmap v703: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:55.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:55 smithi047 ceph-mon[35327]: pgmap v703: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:57.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:57 smithi038 ceph-mon[31227]: pgmap v704: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:57 smithi179 ceph-mon[35781]: pgmap v704: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:57.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:57 smithi047 ceph-mon[35327]: pgmap v704: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:52:59.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:52:59 smithi038 ceph-mon[31227]: pgmap v705: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:59.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:52:59 smithi179 ceph-mon[35781]: pgmap v705: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:52:59.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:52:59 smithi047 ceph-mon[35327]: pgmap v705: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:01.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:01 smithi038 ceph-mon[31227]: pgmap v706: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:01.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:01 smithi179 ceph-mon[35781]: pgmap v706: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:01.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:01 smithi047 ceph-mon[35327]: pgmap v706: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:03.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:03 smithi038 ceph-mon[31227]: pgmap v707: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:03 smithi179 ceph-mon[35781]: pgmap v707: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:03.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:03 smithi047 ceph-mon[35327]: pgmap v707: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:05.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:05 smithi038 ceph-mon[31227]: pgmap v708: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:05 smithi179 ceph-mon[35781]: pgmap v708: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:05.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:05 smithi047 ceph-mon[35327]: pgmap v708: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:07.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:07 smithi038 ceph-mon[31227]: pgmap v709: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:07 smithi179 ceph-mon[35781]: pgmap v709: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:07.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:07 smithi047 ceph-mon[35327]: pgmap v709: 129 pgs: 129 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:09 smithi038 ceph-mon[31227]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:09.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:09 smithi179 ceph-mon[35781]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:09.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:09 smithi047 ceph-mon[35327]: pgmap v710: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:11.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:11 smithi038 ceph-mon[31227]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:11 smithi179 ceph-mon[35781]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:11.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:11 smithi047 ceph-mon[35327]: pgmap v711: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:13 smithi038 ceph-mon[31227]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:13.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:13 smithi179 ceph-mon[35781]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:13.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:13 smithi047 ceph-mon[35327]: pgmap v712: 129 pgs: 129 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:15.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: pgmap v713: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:53:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: pgmap v713: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:53:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: pgmap v713: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:15.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:53:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:53:16.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:53:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:53:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:16.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:53:16.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:53:16.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:53:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:17 smithi179 ceph-mon[35781]: pgmap v714: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:53:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:17 smithi047 ceph-mon[35327]: pgmap v714: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:53:17.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:17 smithi038 ceph-mon[31227]: pgmap v714: 129 pgs: 129 active+clean; 5.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:53:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:19 smithi047 ceph-mon[35327]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:19.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:19 smithi038 ceph-mon[31227]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:19 smithi179 ceph-mon[35781]: pgmap v715: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:21 smithi047 ceph-mon[35327]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:21 smithi038 ceph-mon[31227]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:21 smithi179 ceph-mon[35781]: pgmap v716: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:23.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:23 smithi047 ceph-mon[35327]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:23.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:23 smithi038 ceph-mon[31227]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:23 smithi179 ceph-mon[35781]: pgmap v717: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:25.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:25 smithi047 ceph-mon[35327]: pgmap v718: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:25.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:25 smithi038 ceph-mon[31227]: pgmap v718: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:25 smithi179 ceph-mon[35781]: pgmap v718: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:27.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:27 smithi047 ceph-mon[35327]: pgmap v719: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:27 smithi038 ceph-mon[31227]: pgmap v719: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:27 smithi179 ceph-mon[35781]: pgmap v719: 129 pgs: 129 active+clean; 5.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:29.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:29 smithi047 ceph-mon[35327]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:29.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:29 smithi179 ceph-mon[35781]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:29 smithi038 ceph-mon[31227]: pgmap v720: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:31 smithi038 ceph-mon[31227]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:31 smithi179 ceph-mon[35781]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:31.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:31 smithi047 ceph-mon[35327]: pgmap v721: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:33 smithi038 ceph-mon[31227]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:33 smithi179 ceph-mon[35781]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:33 smithi047 ceph-mon[35327]: pgmap v722: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:35 smithi038 ceph-mon[31227]: pgmap v723: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:35 smithi179 ceph-mon[35781]: pgmap v723: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:35.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:35 smithi047 ceph-mon[35327]: pgmap v723: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:37 smithi038 ceph-mon[31227]: pgmap v724: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:37 smithi179 ceph-mon[35781]: pgmap v724: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:37 smithi047 ceph-mon[35327]: pgmap v724: 129 pgs: 129 active+clean; 5.5 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:39 smithi038 ceph-mon[31227]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:39 smithi179 ceph-mon[35781]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:39.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:39 smithi047 ceph-mon[35327]: pgmap v725: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:41 smithi038 ceph-mon[31227]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:41 smithi179 ceph-mon[35781]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:41 smithi047 ceph-mon[35327]: pgmap v726: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:43.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:43 smithi038 ceph-mon[31227]: pgmap v727: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:43 smithi179 ceph-mon[35781]: pgmap v727: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:43.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:43 smithi047 ceph-mon[35327]: pgmap v727: 129 pgs: 129 active+clean; 5.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:375/600: open fd = 3 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:376/600: open fd = 3 2024-09-19T05:53:45.338 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:377/600: open fd = 3 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:378/600: open fd = 3 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:379/600: open fd = 3 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.339 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:380/600: open fd = 3 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:381/600: open fd = 3 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:382/600: open fd = 3 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.340 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:383/600: open fd = 3 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:384/600: open fd = 3 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:385/600: open fd = 3 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.341 INFO:tasks.workunit.client.0.smithi038.stdout:386/600: open fd = 3 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:387/600: open fd = 3 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:388/600: open fd = 3 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:389/600: open fd = 3 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.342 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:390/600: open fd = 3 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.343 INFO:tasks.workunit.client.0.smithi038.stdout:391/600: open fd = 3 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:392/600: open fd = 3 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:393/600: open fd = 3 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:394/600: open fd = 3 2024-09-19T05:53:45.344 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:395/600: open fd = 3 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:396/600: open fd = 3 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:397/600: open fd = 3 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.345 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:398/600: open fd = 3 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:399/600: open fd = 3 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:400/600: open fd = 3 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.346 INFO:tasks.workunit.client.0.smithi038.stdout:401/600: open fd = 3 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:402/600: open fd = 3 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:403/600: open fd = 3 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:404/600: open fd = 3 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.347 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:405/600: open fd = 3 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:406/600: open fd = 3 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:407/600: open fd = 3 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.348 INFO:tasks.workunit.client.0.smithi038.stdout:408/600: open fd = 3 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:409/600: open fd = 3 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:410/600: open fd = 3 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.349 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:411/600: open fd = 3 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:412/600: open fd = 3 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:413/600: open fd = 3 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.350 INFO:tasks.workunit.client.0.smithi038.stdout:414/600: open fd = 3 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:415/600: open fd = 3 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:416/600: open fd = 3 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:417/600: open fd = 3 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.351 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:418/600: open fd = 3 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:419/600: open fd = 3 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:420/600: open fd = 3 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.352 INFO:tasks.workunit.client.0.smithi038.stdout:421/600: open fd = 3 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:422/600: open fd = 3 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:423/600: open fd = 3 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.353 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:424/600: open fd = 3 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:425/600: open fd = 3 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:426/600: open fd = 3 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.354 INFO:tasks.workunit.client.0.smithi038.stdout:427/600: open fd = 3 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:428/600: open fd = 3 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:429/600: open fd = 3 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:430/600: open fd = 3 2024-09-19T05:53:45.355 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:431/600: open fd = 3 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:432/600: open fd = 3 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:433/600: open fd = 3 2024-09-19T05:53:45.356 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:434/600: open fd = 3 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:435/600: open fd = 3 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:436/600: open fd = 3 2024-09-19T05:53:45.357 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.358 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:53:45.358 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:53:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:45 smithi038 ceph-mon[31227]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:45 smithi179 ceph-mon[35781]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:45 smithi047 ceph-mon[35327]: pgmap v728: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:53:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:47 smithi038 ceph-mon[31227]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:47 smithi179 ceph-mon[35781]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:47 smithi047 ceph-mon[35327]: pgmap v729: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:53:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:49 smithi038 ceph-mon[31227]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:49 smithi179 ceph-mon[35781]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:49.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:49 smithi047 ceph-mon[35327]: pgmap v730: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:51.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:51 smithi038 ceph-mon[31227]: pgmap v731: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:51 smithi179 ceph-mon[35781]: pgmap v731: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:51 smithi047 ceph-mon[35327]: pgmap v731: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:53:53.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:53 smithi038 ceph-mon[31227]: pgmap v732: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:53 smithi179 ceph-mon[35781]: pgmap v732: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:53.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:53 smithi047 ceph-mon[35327]: pgmap v732: 129 pgs: 129 active+clean; 5.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:55 smithi038 ceph-mon[31227]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:53:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:55 smithi179 ceph-mon[35781]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:53:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:55 smithi047 ceph-mon[35327]: pgmap v733: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:53:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:57 smithi038 ceph-mon[31227]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:57 smithi179 ceph-mon[35781]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:57.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:57 smithi047 ceph-mon[35327]: pgmap v734: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:53:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:53:59 smithi038 ceph-mon[31227]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:53:59 smithi179 ceph-mon[35781]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:53:59.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:53:59 smithi047 ceph-mon[35327]: pgmap v735: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:54:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:01 smithi038 ceph-mon[31227]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:54:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:01 smithi179 ceph-mon[35781]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:54:01.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:01 smithi047 ceph-mon[35327]: pgmap v736: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:54:03.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:03 smithi038 ceph-mon[31227]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:54:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:03 smithi179 ceph-mon[35781]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:54:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:03 smithi047 ceph-mon[35327]: pgmap v737: 129 pgs: 129 active+clean; 5.8 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:54:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:05 smithi038 ceph-mon[31227]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:54:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:05 smithi179 ceph-mon[35781]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:54:05.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:05 smithi047 ceph-mon[35327]: pgmap v738: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:54:06.866 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:54:06.866 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T05:54:06.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:54:07.139 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:07.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 -- 172.21.15.38:0/4192410116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8428101df0 msgr2=0x7f84281021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 --2- 172.21.15.38:0/4192410116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8428101df0 0x7f84281021d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8418009900 tx=0x7f841802a410 comp rx=0 tx=0).stop 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 -- 172.21.15.38:0/4192410116 shutdown_connections 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 --2- 172.21.15.38:0/4192410116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428106f20 0x7f842810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 --2- 172.21.15.38:0/4192410116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281027a0 0x7f84281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 --2- 172.21.15.38:0/4192410116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8428101df0 0x7f84281021d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 -- 172.21.15.38:0/4192410116 >> 172.21.15.38:0/4192410116 conn(0x7f84280fd8f0 msgr2=0x7f84280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 -- 172.21.15.38:0/4192410116 shutdown_connections 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.640+0000 7f842e70d640 1 -- 172.21.15.38:0/4192410116 wait complete. 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 Processor -- start 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 -- start start 2024-09-19T05:54:07.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428101df0 0x7f84280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 0x7f8428074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8428111f80 con 0x7f84281027a0 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8428111e00 con 0x7f8428106f20 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842e70d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8428112100 con 0x7f8428101df0 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51574/0 (socket says 172.21.15.38:51574) 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 -- 172.21.15.38:0/3068870212 learned_addr learned my addr 172.21.15.38:0/3068870212 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f8427fff640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428101df0 0x7f84280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f842ccaa640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 0x7f8428074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 -- 172.21.15.38:0/3068870212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428101df0 msgr2=0x7f84280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428101df0 0x7f84280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 -- 172.21.15.38:0/3068870212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 msgr2=0x7f8428074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 0x7f8428074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.641+0000 7f84277fe640 1 -- 172.21.15.38:0/3068870212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84281a6840 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f842ccaa640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 0x7f8428074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f84277fe640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c920 tx=0x7f841c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c007c70 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281a6b30 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c007e10 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c018890 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.642+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a70c0 con 0x7f84281027a0 2024-09-19T05:54:07.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.643+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84281021d0 con 0x7f84281027a0 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.644+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f841c018a70 con 0x7f84281027a0 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.645+0000 7f84257fa640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.645+0000 7f8427fff640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.645+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f841c0cfe20 con 0x7f84281027a0 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.645+0000 7f8427fff640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 0x7f84100a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f84180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:07.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.649+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f841c014030 con 0x7f84281027a0 2024-09-19T05:54:07.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.864+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8428109b70 con 0x7f84281027a0 2024-09-19T05:54:07.866 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:07 smithi038 ceph-mon[31227]: pgmap v739: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:54:07.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.864+0000 7f84257fa640 1 -- 172.21.15.38:0/3068870212 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f841c09c780 con 0x7f84281027a0 2024-09-19T05:54:07.867 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:54:07.868 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:54:07.868 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 msgr2=0x7f84100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 0x7f84100a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8418009230 tx=0x7f84180058d0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 msgr2=0x7f8428074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f841c00c920 tx=0x7f841c00cde0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 shutdown_connections 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84100a3240 0x7f84100a5700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428106f20 0x7f8428074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84281027a0 0x7f8428074a30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 --2- 172.21.15.38:0/3068870212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428101df0 0x7f84280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:07.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 >> 172.21.15.38:0/3068870212 conn(0x7f84280fd8f0 msgr2=0x7f84280fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:07.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.867+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 shutdown_connections 2024-09-19T05:54:07.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:07.868+0000 7f842e70d640 1 -- 172.21.15.38:0/3068870212 wait complete. 2024-09-19T05:54:07.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:07 smithi179 ceph-mon[35781]: pgmap v739: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:54:07.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:07 smithi047 ceph-mon[35327]: pgmap v739: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:54:08.632 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T05:54:08.633 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T05:54:08.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:08 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3068870212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:08.895 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:08.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:08 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3068870212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:08.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:08 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3068870212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:09.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 -- 172.21.15.38:0/1400812854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 msgr2=0x7ff24410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 --2- 172.21.15.38:0/1400812854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24410ea90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009ca0 tx=0x7ff23c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 -- 172.21.15.38:0/1400812854 shutdown_connections 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 --2- 172.21.15.38:0/1400812854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24410ea90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 --2- 172.21.15.38:0/1400812854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 0x7ff2441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 --2- 172.21.15.38:0/1400812854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 0x7ff2441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 -- 172.21.15.38:0/1400812854 >> 172.21.15.38:0/1400812854 conn(0x7ff2440fd8c0 msgr2=0x7ff2440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 -- 172.21.15.38:0/1400812854 shutdown_connections 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.401+0000 7ff249d83640 1 -- 172.21.15.38:0/1400812854 wait complete. 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 Processor -- start 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 -- start start 2024-09-19T05:54:09.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 0x7ff244198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 0x7ff2441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.402+0000 7ff249d83640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff244111f90 con 0x7ff244106f10 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff249d83640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff244111e10 con 0x7ff244102790 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff249d83640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff244112110 con 0x7ff244101de0 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51598/0 (socket says 172.21.15.38:51598) 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 -- 172.21.15.38:0/63058650 learned_addr learned my addr 172.21.15.38:0/63058650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:09.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff242ffd640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 0x7ff2441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff2437fe640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 0x7ff244198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 -- 172.21.15.38:0/63058650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 msgr2=0x7ff244198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 0x7ff244198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 -- 172.21.15.38:0/63058650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 msgr2=0x7ff2441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 0x7ff2441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 -- 172.21.15.38:0/63058650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24419d420 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff243fff640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031a60 tx=0x7ff23c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff240ff9640 1 -- 172.21.15.38:0/63058650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003c30 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24419d6b0 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff240ff9640 1 -- 172.21.15.38:0/63058650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23c003dd0 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff240ff9640 1 -- 172.21.15.38:0/63058650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0047e0 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.403+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a4ff0 con 0x7ff244106f10 2024-09-19T05:54:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff240ff9640 1 -- 172.21.15.38:0/63058650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff23c02f790 con 0x7ff244106f10 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 Processor -- start 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 -- start start 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x56490547fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5649053d82a0 con 0x564905472760 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56490537fb10 con 0x56490547d940 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff24c15c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5649053d4260 con 0x564905473080 2024-09-19T05:54:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51604/0 (socket says 172.21.15.38:51604) 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 -- 172.21.15.38:0/636323662 learned_addr learned my addr 172.21.15.38:0/636323662 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x56490547fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff2437fe640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 -- 172.21.15.38:0/636323662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 msgr2=0x56490547d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 -- 172.21.15.38:0/636323662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 msgr2=0x56490547fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x56490547fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff240ff9640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 0x7ff22c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff242ffd640 1 -- 172.21.15.38:0/636323662 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564905149b90 con 0x564905472760 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.407+0000 7ff2437fe640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff243fff640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 0x7ff22c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff242ffd640 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff238005d90 tx=0x7ff23800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4143d4d0945bad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff240ff9640 1 -- 172.21.15.38:0/63058650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7ff23c0f11b0 con 0x7ff244106f10 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff229ffb640 1 -- 172.21.15.38:0/636323662 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238018030 con 0x564905472760 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff229ffb640 1 -- 172.21.15.38:0/636323662 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff238005500 con 0x564905472760 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.408+0000 7ff229ffb640 1 -- 172.21.15.38:0/636323662 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23800d7b0 con 0x564905472760 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- 172.21.15.38:0/636323662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 msgr2=0x564905472b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff243fff640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 0x7ff22c0a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009430 tx=0x7ff23c0b28f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff238005d90 tx=0x7ff23800c5e0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- 172.21.15.38:0/636323662 shutdown_connections 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x56490547fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 --2- 172.21.15.38:0/636323662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x564905472b40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- 172.21.15.38:0/636323662 >> 172.21.15.38:0/636323662 conn(0x56490546ecf0 msgr2=0x56490546f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- 172.21.15.38:0/636323662 shutdown_connections 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- 172.21.15.38:0/636323662 wait complete. 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 -- start start 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.409+0000 7ff24c15c740 1 Processor -- start 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905473080 0x56490547ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56490547d940 0x56490547b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564905480510 con 0x564905472760 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5649053606e0 con 0x564905473080 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564905480690 con 0x56490547d940 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff24c15c740 10 client.0 _ll_register_callbacks cb 0x564905384a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51608/0 (socket says 172.21.15.38:51608) 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 -- 172.21.15.38:0/240950863 learned_addr learned my addr 172.21.15.38:0/240950863 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff2437fe640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905473080 0x56490547ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56490547d940 0x56490547b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 -- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56490547d940 msgr2=0x56490547b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56490547d940 0x56490547b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 -- 172.21.15.38:0/240950863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905473080 msgr2=0x56490547ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905473080 0x56490547ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 -- 172.21.15.38:0/240950863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564905489a30 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff242ffd640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff238004cc0 tx=0x7ff238005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f797f5b254cc004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.410+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238004220 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7ff2380043c0 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff238005500 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff238030050 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 10 client.15930 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56490548c330 con 0x564905472760 2024-09-19T05:54:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff238005500 con 0x564905472760 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff238028020 con 0x564905472760 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 10 client.15930 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 20 client.15930 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56490548d730 0x56490548fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 4 client.15930 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.411+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649053e59d0 con 0x56490548d730 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.412+0000 7ff2437fe640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56490548d730 0x56490548fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.412+0000 7ff222ffd640 10 client.15930 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.412+0000 7ff2437fe640 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56490548d730 0x56490548fb20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5b17c4b98024b7 server_cookie=686559fe8a5af265 in_seq=0 out_seq=0 2024-09-19T05:54:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff222ffd640 1 -- 172.21.15.38:0/240950863 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649053e59d0 con 0x56490548d730 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff222ffd640 10 client.15930 handle_command_reply: tid=0 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 client.15930 shutdown 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 msgr2=0x56490547a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff238004cc0 tx=0x7ff238005f40 comp rx=0 tx=0).stop 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 shutdown_connections 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56490548d730 0x56490548fb20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56490547d940 0x56490547b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905473080 0x56490547ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 --2- 172.21.15.38:0/240950863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x56490547a850 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 >> 172.21.15.38:0/240950863 conn(0x56490546ecf0 msgr2=0x564905474a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff222ffd640 0 client.15930 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:09.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 shutdown_connections 2024-09-19T05:54:09.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 1 -- 172.21.15.38:0/240950863 wait complete. 2024-09-19T05:54:09.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.413+0000 7ff24c15c740 20 client.15930 trim_cache size 0 max 16384 2024-09-19T05:54:09.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 Processor -- start 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 -- start start 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905472760 0x564905479b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441aed60 con 0x56490547d940 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441b08a0 con 0x564905472760 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff24c15c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441af430 con 0x564905473080 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51616/0 (socket says 172.21.15.38:51616) 2024-09-19T05:54:09.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.430+0000 7ff242ffd640 1 -- 172.21.15.38:0/2059599311 learned_addr learned my addr 172.21.15.38:0/2059599311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905472760 0x564905479b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff2437fe640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 -- 172.21.15.38:0/2059599311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 msgr2=0x56490547c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 -- 172.21.15.38:0/2059599311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905472760 msgr2=0x564905479b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905472760 0x564905479b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 -- 172.21.15.38:0/2059599311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564905472eb0 con 0x56490547d940 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff2437fe640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff242ffd640 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff23800c610 tx=0x7ff238011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff58e6104da9350 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff2297fa640 1 -- 172.21.15.38:0/2059599311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238018040 con 0x56490547d940 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff2297fa640 1 -- 172.21.15.38:0/2059599311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff238004cf0 con 0x56490547d940 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.431+0000 7ff2297fa640 1 -- 172.21.15.38:0/2059599311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23800dc60 con 0x56490547d940 2024-09-19T05:54:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- 172.21.15.38:0/2059599311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 msgr2=0x56490547af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff23800c610 tx=0x7ff238011040 comp rx=0 tx=0).stop 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- 172.21.15.38:0/2059599311 shutdown_connections 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56490547d940 0x56490547af20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x56490547c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 --2- 172.21.15.38:0/2059599311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564905472760 0x564905479b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- 172.21.15.38:0/2059599311 >> 172.21.15.38:0/2059599311 conn(0x56490546ecf0 msgr2=0x56490547a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- 172.21.15.38:0/2059599311 shutdown_connections 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- 172.21.15.38:0/2059599311 wait complete. 2024-09-19T05:54:09.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 -- start start 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.432+0000 7ff24c15c740 1 Processor -- start 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x5649054814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x564905481c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441b01d0 con 0x564905472760 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441afb00 con 0x56490547d940 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441acd40 con 0x564905473080 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff24c15c740 10 client.0 _ll_register_callbacks cb 0x7ff238004ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:09.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51622/0 (socket says 172.21.15.38:51622) 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 -- 172.21.15.38:0/3155969458 learned_addr learned my addr 172.21.15.38:0/3155969458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff2437fe640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x5649054814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x564905481c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 -- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 msgr2=0x5649054814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x5649054814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 -- 172.21.15.38:0/3155969458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 msgr2=0x564905481c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x564905481c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 -- 172.21.15.38:0/3155969458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649054f7260 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff242ffd640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff23802d040 tx=0x7ff23800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940c19ddfcbe7793 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.433+0000 7ff222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238018050 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7ff238005660 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff238004cf0 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff238045050 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649054f8340 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff238004cf0 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff238028070 con 0x564905472760 2024-09-19T05:54:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 10 client.15936 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 20 client.15936 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649054f8960 0x5649054fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 4 client.15936 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.434+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5649053dfd20 con 0x5649054f8960 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.435+0000 7ff2437fe640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649054f8960 0x5649054fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.435+0000 7ff222ffd640 10 client.15936 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.435+0000 7ff2437fe640 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649054f8960 0x5649054fad50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7777e726e79314 server_cookie=5bfac8376af3ab7d in_seq=0 out_seq=0 2024-09-19T05:54:09.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff222ffd640 1 -- 172.21.15.38:0/3155969458 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649053dfd20 con 0x5649054f8960 2024-09-19T05:54:09.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff222ffd640 10 client.15936 handle_command_reply: tid=0 2024-09-19T05:54:09.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 client.15936 shutdown 2024-09-19T05:54:09.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 msgr2=0x5649054861b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff23802d040 tx=0x7ff23800c330 comp rx=0 tx=0).stop 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 shutdown_connections 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5649054f8960 0x5649054fad50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56490547d940 0x564905481c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564905473080 0x5649054814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 --2- 172.21.15.38:0/3155969458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564905472760 0x5649054861b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff222ffd640 0 client.15936 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 >> 172.21.15.38:0/3155969458 conn(0x56490546ecf0 msgr2=0x56490547b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 shutdown_connections 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 1 -- 172.21.15.38:0/3155969458 wait complete. 2024-09-19T05:54:09.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.446+0000 7ff24c15c740 20 client.15936 trim_cache size 0 max 16384 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 42, 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "2a2a234b": { 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 42, 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:54:09.451 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 142.2, 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:54:09.452 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:54:09.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 msgr2=0x7ff22c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 0x7ff22c0a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009430 tx=0x7ff23c0b28f0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 msgr2=0x7ff24419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031a60 tx=0x7ff23c031a90 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 shutdown_connections 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7ff22c0a3240 0x7ff22c0a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff244106f10 0x7ff24419cd40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff244102790 0x7ff2441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 --2- 172.21.15.38:0/63058650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff244101de0 0x7ff244198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 >> 172.21.15.38:0/63058650 conn(0x7ff2440fd8c0 msgr2=0x7ff2440fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 shutdown_connections 2024-09-19T05:54:09.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:09.447+0000 7ff249d83640 1 -- 172.21.15.38:0/63058650 wait complete. 2024-09-19T05:54:09.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:09 smithi038 ceph-mon[31227]: pgmap v740: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:54:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:09 smithi179 ceph-mon[35781]: pgmap v740: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:54:09.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:09 smithi047 ceph-mon[35327]: pgmap v740: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:54:10.152 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2a2a234b', quiesce took 3.3 seconds. Will release after: 18.9 2024-09-19T05:54:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:11 smithi038 ceph-mon[31227]: pgmap v741: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:11 smithi179 ceph-mon[35781]: pgmap v741: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:11.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:11 smithi047 ceph-mon[35327]: pgmap v741: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:13 smithi038 ceph-mon[31227]: pgmap v742: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:13 smithi179 ceph-mon[35781]: pgmap v742: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:13.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:13 smithi047 ceph-mon[35327]: pgmap v742: 129 pgs: 129 active+clean; 5.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:54:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:15 smithi038 ceph-mon[31227]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:54:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:15 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:15 smithi179 ceph-mon[35781]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:54:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:15 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:15.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:15 smithi047 ceph-mon[35327]: pgmap v743: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-09-19T05:54:15.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:15 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:54:16.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:54:16.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:54:16.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:16 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:54:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:54:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:54:16.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:16 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:54:16.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:54:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:54:16.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:16 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:54:17.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:17 smithi038 ceph-mon[31227]: pgmap v744: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-19T05:54:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:17 smithi179 ceph-mon[35781]: pgmap v744: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-19T05:54:17.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:17 smithi047 ceph-mon[35327]: pgmap v744: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-19T05:54:19.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:19 smithi038 ceph-mon[31227]: pgmap v745: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-19T05:54:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:19 smithi179 ceph-mon[35781]: pgmap v745: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-19T05:54:19.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:19 smithi047 ceph-mon[35327]: pgmap v745: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-19T05:54:21.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:21 smithi038 ceph-mon[31227]: pgmap v746: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:21 smithi179 ceph-mon[35781]: pgmap v746: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:21.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:21 smithi047 ceph-mon[35327]: pgmap v746: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:23 smithi038 ceph-mon[31227]: pgmap v747: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:23 smithi179 ceph-mon[35781]: pgmap v747: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:23.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:23 smithi047 ceph-mon[35327]: pgmap v747: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:25 smithi179 ceph-mon[35781]: pgmap v748: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:25 smithi047 ceph-mon[35327]: pgmap v748: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:26.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:25 smithi038 ceph-mon[31227]: pgmap v748: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:54:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:27 smithi179 ceph-mon[35781]: pgmap v749: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:54:27.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:27 smithi047 ceph-mon[35327]: pgmap v749: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:54:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:27 smithi038 ceph-mon[31227]: pgmap v749: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-19T05:54:29.053 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:54:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:29.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:29 smithi038 ceph-mon[31227]: pgmap v750: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:54:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.868+0000 7f1549436640 1 -- 172.21.15.38:0/3585562602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 msgr2=0x7f15441006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.868+0000 7f1549436640 1 --2- 172.21.15.38:0/3585562602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f15441006d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f153402a420 comp rx=0 tx=0).stop 2024-09-19T05:54:29.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 -- 172.21.15.38:0/3585562602 shutdown_connections 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 --2- 172.21.15.38:0/3585562602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 --2- 172.21.15.38:0/3585562602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 0x7f1544106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 --2- 172.21.15.38:0/3585562602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f15441006d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 -- 172.21.15.38:0/3585562602 >> 172.21.15.38:0/3585562602 conn(0x7f1544077210 msgr2=0x7f15440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 -- 172.21.15.38:0/3585562602 shutdown_connections 2024-09-19T05:54:29.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 -- 172.21.15.38:0/3585562602 wait complete. 2024-09-19T05:54:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 Processor -- start 2024-09-19T05:54:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.869+0000 7f1549436640 1 -- start start 2024-09-19T05:54:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 0x7f154419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1544111fa0 con 0x7f15441002f0 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1544111e20 con 0x7f1544100ca0 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1549436640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1544112120 con 0x7f1544106f50 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36496/0 (socket says 172.21.15.38:36496) 2024-09-19T05:54:29.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 -- 172.21.15.38:0/3486356058 learned_addr learned my addr 172.21.15.38:0/3486356058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f15427fc640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 0x7f154419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f15437fe640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 -- 172.21.15.38:0/3486356058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 msgr2=0x7f154419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 -- 172.21.15.38:0/3486356058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 msgr2=0x7f154419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 0x7f154419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 -- 172.21.15.38:0/3486356058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154419f6a0 con 0x7f15441002f0 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f15437fe640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f1542ffd640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1534009230 tx=0x7f1534004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.870+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153403f030 con 0x7f15441002f0 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.871+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154419f930 con 0x7f15441002f0 2024-09-19T05:54:29.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.871+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1534003ad0 con 0x7f15441002f0 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.871+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534035070 con 0x7f15441002f0 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.871+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15441a7270 con 0x7f15441002f0 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.872+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15441006d0 con 0x7f15441002f0 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.873+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1534042030 con 0x7f15441002f0 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.873+0000 7f152bfff640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 0x7f152c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:29.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.874+0000 7f15427fc640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 0x7f152c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.874+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f15340f01e0 con 0x7f15441002f0 2024-09-19T05:54:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.874+0000 7f15427fc640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 0x7f152c0a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1544106bf0 tx=0x7f153800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:29.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:29.878+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15340bcb40 con 0x7f15441002f0 2024-09-19T05:54:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:29 smithi179 ceph-mon[35781]: pgmap v750: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:54:29.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:29 smithi047 ceph-mon[35327]: pgmap v750: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-19T05:54:30.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.091+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f154410e550 con 0x7f15441002f0 2024-09-19T05:54:30.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.092+0000 7f152bfff640 1 -- 172.21.15.38:0/3486356058 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f15340bc440 con 0x7f15441002f0 2024-09-19T05:54:30.095 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:54:30.095 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:54:30.095 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.094+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 msgr2=0x7f152c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.094+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 0x7f152c0a5700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1544106bf0 tx=0x7f153800b390 comp rx=0 tx=0).stop 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.094+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 msgr2=0x7f154419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.094+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1534009230 tx=0x7f1534004270 comp rx=0 tx=0).stop 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 shutdown_connections 2024-09-19T05:54:30.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f152c0a3240 0x7f152c0a5700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1544106f50 0x7f154419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1544100ca0 0x7f154419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 --2- 172.21.15.38:0/3486356058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15441002f0 0x7f154419a6f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 >> 172.21.15.38:0/3486356058 conn(0x7f1544077210 msgr2=0x7f15441054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 shutdown_connections 2024-09-19T05:54:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:30.095+0000 7f1549436640 1 -- 172.21.15.38:0/3486356058 wait complete. 2024-09-19T05:54:30.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:30 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3486356058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:30.922 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 2a2a234b --release --await' 2024-09-19T05:54:30.922 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 2a2a234b --release --await 2024-09-19T05:54:30.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:30 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3486356058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:30.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:30 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3486356058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:31.187 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:31.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 -- 172.21.15.38:0/201864955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20102790 msgr2=0x7f1e201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 --2- 172.21.15.38:0/201864955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20102790 0x7f1e201067e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009900 tx=0x7f1e0802a410 comp rx=0 tx=0).stop 2024-09-19T05:54:31.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 -- 172.21.15.38:0/201864955 shutdown_connections 2024-09-19T05:54:31.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 --2- 172.21.15.38:0/201864955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20106f10 0x7f1e2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 --2- 172.21.15.38:0/201864955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20102790 0x7f1e201067e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 --2- 172.21.15.38:0/201864955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 0x7f1e201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 -- 172.21.15.38:0/201864955 >> 172.21.15.38:0/201864955 conn(0x7f1e200fd8c0 msgr2=0x7f1e200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 -- 172.21.15.38:0/201864955 shutdown_connections 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.718+0000 7f1e24f39640 1 -- 172.21.15.38:0/201864955 wait complete. 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 Processor -- start 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 -- start start 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 0x7f1e20198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20102790 0x7f1e20198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20111f70 con 0x7f1e20106f10 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20111df0 con 0x7f1e20101de0 2024-09-19T05:54:31.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.719+0000 7f1e24f39640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201120f0 con 0x7f1e20102790 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36502/0 (socket says 172.21.15.38:36502) 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 -- 172.21.15.38:0/3354826843 learned_addr learned my addr 172.21.15.38:0/3354826843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 0x7f1e20198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20102790 0x7f1e20198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 -- 172.21.15.38:0/3354826843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20102790 msgr2=0x7f1e20198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20102790 0x7f1e20198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 -- 172.21.15.38:0/3354826843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 msgr2=0x7f1e20198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 0x7f1e20198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 -- 172.21.15.38:0/3354826843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2019d3b0 con 0x7f1e20106f10 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e1ed9d640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800eaa0 tx=0x7f1e1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e177fe640 1 -- 172.21.15.38:0/3354826843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800cdb0 con 0x7f1e20106f10 2024-09-19T05:54:31.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e177fe640 1 -- 172.21.15.38:0/3354826843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18005020 con 0x7f1e20106f10 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2019d6a0 con 0x7f1e20106f10 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e177fe640 1 -- 172.21.15.38:0/3354826843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180116b0 con 0x7f1e20106f10 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.720+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a4f80 con 0x7f1e20106f10 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.722+0000 7f1e177fe640 1 -- 172.21.15.38:0/3354826843 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1e18005560 con 0x7f1e20106f10 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 Processor -- start 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 -- start start 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca02440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 0x56209ca0cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56209c967620 con 0x56209ca0d240 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56209c90ee50 con 0x56209ca02060 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e27312740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56209c9635e0 con 0x56209ca02980 2024-09-19T05:54:31.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e177fe640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 0x7f1e0c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36516/0 (socket says 172.21.15.38:36516) 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.723+0000 7f1e17fff640 1 -- 172.21.15.38:0/493861380 learned_addr learned my addr 172.21.15.38:0/493861380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e177fe640 1 -- 172.21.15.38:0/3354826843 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f1e180cfec0 con 0x7f1e20106f10 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e1ed9d640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 0x7f1e0c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca02440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 0x56209ca0cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 -- 172.21.15.38:0/493861380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 msgr2=0x56209ca0cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 0x56209ca0cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 -- 172.21.15.38:0/493861380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 msgr2=0x56209ca02440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca02440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 -- 172.21.15.38:0/493861380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209c8ffd40 con 0x56209ca0d240 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 0x56209ca0cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e17fff640 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1e08031550 tx=0x7f1e08005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8462468e8ebafaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e1ed9d640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 0x7f1e0c0a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1e18098af0 tx=0x7f1e18000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e14ff9640 1 -- 172.21.15.38:0/493861380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08035040 con 0x56209ca0d240 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e14ff9640 1 -- 172.21.15.38:0/493861380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08004030 con 0x56209ca0d240 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.724+0000 7f1e14ff9640 1 -- 172.21.15.38:0/493861380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080039e0 con 0x56209ca0d240 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- 172.21.15.38:0/493861380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 msgr2=0x56209ca0f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1e08031550 tx=0x7f1e08005b50 comp rx=0 tx=0).stop 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- 172.21.15.38:0/493861380 shutdown_connections 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca0f630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02980 0x56209ca0cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 --2- 172.21.15.38:0/493861380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca02440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- 172.21.15.38:0/493861380 >> 172.21.15.38:0/493861380 conn(0x56209c9fe5f0 msgr2=0x56209c9fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- 172.21.15.38:0/493861380 shutdown_connections 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- 172.21.15.38:0/493861380 wait complete. 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.725+0000 7f1e27312740 1 -- start start 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 Processor -- start 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca0d240 0x56209ca0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56209ca0fe10 con 0x56209ca02980 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56209c8eff10 con 0x56209ca0d240 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56209ca0ff90 con 0x56209ca02060 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e27312740 10 client.0 _ll_register_callbacks cb 0x56209c913db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36530/0 (socket says 172.21.15.38:36530) 2024-09-19T05:54:31.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 -- 172.21.15.38:0/3384668805 learned_addr learned my addr 172.21.15.38:0/3384668805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca0d240 0x56209ca0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 -- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 msgr2=0x56209ca0a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca0a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 -- 172.21.15.38:0/3384668805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca0d240 msgr2=0x56209ca0abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca0d240 0x56209ca0abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 -- 172.21.15.38:0/3384668805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209c676580 con 0x56209ca02980 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1e100137c0 tx=0x7f1e10013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79e8ed536ef53c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.726+0000 7f1e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10020060 con 0x56209ca02980 2024-09-19T05:54:31.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e08004030 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f1e10011dd0 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e10027050 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56209c740960 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08004030 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1e1001a030 con 0x56209ca02980 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 10 client.15954 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 20 client.15954 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca1ce60 0x56209ca1f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 4 client.15954 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.727+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56209c974b90 con 0x56209ca1ce60 2024-09-19T05:54:31.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.728+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca1ce60 0x56209ca1f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.728+0000 7f1e05ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.728+0000 7f1e17fff640 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca1ce60 0x56209ca1f280 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1fbbcdfa46eb10b server_cookie=ecb4eda101e9d87a in_seq=0 out_seq=0 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e05ffb640 1 -- 172.21.15.38:0/3384668805 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56209c974b90 con 0x56209ca1ce60 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e05ffb640 10 client.15954 handle_command_reply: tid=0 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 client.15954 shutdown 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 msgr2=0x56209ca0a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1e100137c0 tx=0x7f1e10013c80 comp rx=0 tx=0).stop 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 shutdown_connections 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca1ce60 0x56209ca1f280 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca0d240 0x56209ca0abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca0a670 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 --2- 172.21.15.38:0/3384668805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 >> 172.21.15.38:0/3384668805 conn(0x56209c9fe5f0 msgr2=0x56209ca042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e05ffb640 0 client.15954 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 shutdown_connections 2024-09-19T05:54:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 1 -- 172.21.15.38:0/3384668805 wait complete. 2024-09-19T05:54:31.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.729+0000 7f1e27312740 20 client.15954 trim_cache size 0 max 16384 2024-09-19T05:54:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.746+0000 7f1e27312740 1 Processor -- start 2024-09-19T05:54:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 -- start start 2024-09-19T05:54:31.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca24b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 0x56209ca25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201aecf0 con 0x56209ca0d240 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201b0830 con 0x56209ca02980 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e27312740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201af3c0 con 0x56209ca02060 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36538/0 (socket says 172.21.15.38:36538) 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 -- 172.21.15.38:0/862840131 learned_addr learned my addr 172.21.15.38:0/862840131 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca24b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 0x56209ca25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 -- 172.21.15.38:0/862840131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 msgr2=0x56209ca24b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca24b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 -- 172.21.15.38:0/862840131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 msgr2=0x56209ca25090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 0x56209ca25090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e17fff640 1 -- 172.21.15.38:0/862840131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209ca1f900 con 0x56209ca0d240 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.747+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 0x56209ca25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e17fff640 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009900 tx=0x7f1e08005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7171f5301b6a2c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e07fff640 1 -- 172.21.15.38:0/862840131 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08035040 con 0x56209ca0d240 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e07fff640 1 -- 172.21.15.38:0/862840131 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08053380 con 0x56209ca0d240 2024-09-19T05:54:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e07fff640 1 -- 172.21.15.38:0/862840131 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080038f0 con 0x56209ca0d240 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e27312740 1 -- 172.21.15.38:0/862840131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 msgr2=0x56209ca073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.748+0000 7f1e27312740 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009900 tx=0x7f1e08005b50 comp rx=0 tx=0).stop 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- 172.21.15.38:0/862840131 shutdown_connections 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca0d240 0x56209ca073b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02980 0x56209ca25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- 172.21.15.38:0/862840131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca02060 0x56209ca24b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- 172.21.15.38:0/862840131 >> 172.21.15.38:0/862840131 conn(0x56209c9fe5f0 msgr2=0x56209ca08a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- 172.21.15.38:0/862840131 shutdown_connections 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- 172.21.15.38:0/862840131 wait complete. 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- start start 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 Processor -- start 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca0d240 0x56209ca121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201b0160 con 0x56209ca02980 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201afa90 con 0x56209ca02060 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201accd0 con 0x56209ca0d240 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e27312740 10 client.0 _ll_register_callbacks cb 0x7f1e10009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e1e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36544/0 (socket says 172.21.15.38:36544) 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.749+0000 7f1e1e59c640 1 -- 172.21.15.38:0/1996549119 learned_addr learned my addr 172.21.15.38:0/1996549119 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e17fff640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e17fff640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca0d240 0x56209ca121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 -- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca0d240 msgr2=0x56209ca121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca0d240 0x56209ca121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 -- 172.21.15.38:0/1996549119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 msgr2=0x56209ca151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 -- 172.21.15.38:0/1996549119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209ca127a0 con 0x56209ca02980 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e17fff640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e1e59c640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000c00 tx=0x7f1e10011830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d255710686d2db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:31.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000eb20 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f1e10011bd0 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e08053380 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e10020020 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e27312740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.750+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56209ca88450 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e08053380 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1e10033030 con 0x56209ca02980 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 10 client.15960 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 20 client.15960 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca88c40 0x56209ca8b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 4 client.15960 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2a2a234b", "release": true, "await": true}] 2024-09-19T05:54:31.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2a2a234b", "release": true, "await": true}) -- 0x56209c96f0a0 con 0x56209ca88c40 2024-09-19T05:54:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.751+0000 7f1e17fff640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca88c40 0x56209ca8b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.752+0000 7f1e17fff640 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca88c40 0x56209ca8b060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472887d0cf38435b server_cookie=39ea2caf928f77ef in_seq=0 out_seq=0 2024-09-19T05:54:31.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.752+0000 7f1e05ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e05ffb640 1 -- 172.21.15.38:0/1996549119 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56209c96f0a0 con 0x56209ca88c40 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e05ffb640 10 client.15960 handle_command_reply: tid=0 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 client.15960 shutdown 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 msgr2=0x56209ca15720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000c00 tx=0x7f1e10011830 comp rx=0 tx=0).stop 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 shutdown_connections 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x56209ca88c40 0x56209ca8b060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56209ca0d240 0x56209ca121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56209ca02980 0x56209ca15720 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 --2- 172.21.15.38:0/1996549119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56209ca02060 0x56209ca151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e05ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 >> 172.21.15.38:0/1996549119 conn(0x56209c9fe5f0 msgr2=0x56209ca03ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 shutdown_connections 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 1 -- 172.21.15.38:0/1996549119 wait complete. 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.755+0000 7f1e27312740 20 client.15960 trim_cache size 0 max 16384 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:54:31.760 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 44, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "2a2a234b": { 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 44, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 142.2, 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:54:31.761 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:31.762 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 msgr2=0x7f1e0c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 0x7f1e0c0a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1e18098af0 tx=0x7f1e18000f40 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 msgr2=0x7f1e2019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800eaa0 tx=0x7f1e1800ef60 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 shutdown_connections 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f1e0c0a32c0 0x7f1e0c0a5780 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1e20106f10 0x7f1e2019ccd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20102790 0x7f1e20198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 --2- 172.21.15.38:0/3354826843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e20101de0 0x7f1e20198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 >> 172.21.15.38:0/3354826843 conn(0x7f1e200fd8c0 msgr2=0x7f1e200fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 shutdown_connections 2024-09-19T05:54:31.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:31.756+0000 7f1e24f39640 1 -- 172.21.15.38:0/3354826843 wait complete. 2024-09-19T05:54:31.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:31 smithi038 ceph-mon[31227]: pgmap v751: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:31.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:31 smithi179 ceph-mon[35781]: pgmap v751: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:32.041 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:31 smithi047 ceph-mon[35327]: pgmap v751: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:32.551 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2a2a234b', total seconds elapsed: 25.7 2024-09-19T05:54:32.551 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T05:54:32.551 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:54:32.810 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:33.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 -- 172.21.15.38:0/1847199161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618103660 msgr2=0x7f161810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 --2- 172.21.15.38:0/1847199161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618103660 0x7f161810bb50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1600009900 tx=0x7f160002a480 comp rx=0 tx=0).stop 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 -- 172.21.15.38:0/1847199161 shutdown_connections 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 --2- 172.21.15.38:0/1847199161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f161810c170 0x7f161810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 --2- 172.21.15.38:0/1847199161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618103660 0x7f161810bb50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 --2- 172.21.15.38:0/1847199161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1618102d40 0x7f1618103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 -- 172.21.15.38:0/1847199161 >> 172.21.15.38:0/1847199161 conn(0x7f16180fc7a0 msgr2=0x7f16180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 -- 172.21.15.38:0/1847199161 shutdown_connections 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.339+0000 7f161ce87640 1 -- 172.21.15.38:0/1847199161 wait complete. 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 Processor -- start 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 -- start start 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:33.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1618103660 0x7f1618100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f161810c170 0x7f1618101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1618112000 con 0x7f1618102d40 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1618111e80 con 0x7f161810c170 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161ce87640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1618112180 con 0x7f1618103660 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36564/0 (socket says 172.21.15.38:36564) 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 -- 172.21.15.38:0/3496907456 learned_addr learned my addr 172.21.15.38:0/3496907456 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f1616d9d640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f161810c170 0x7f1618101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f160ffff640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1618103660 0x7f1618100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 -- 172.21.15.38:0/3496907456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1618103660 msgr2=0x7f1618100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1618103660 0x7f1618100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 -- 172.21.15.38:0/3496907456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f161810c170 msgr2=0x7f1618101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:33.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f161810c170 0x7f1618101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.340+0000 7f161659c640 1 -- 172.21.15.38:0/3496907456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16181a6920 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f161659c640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f160800d950 tx=0x7f160800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608017070 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1608009c30 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16181a6c10 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160800f6b0 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.341+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a7170 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.342+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1618103120 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.343+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f16080109d0 con 0x7f1618102d40 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.344+0000 7f160f7fe640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 0x7f16040a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.344+0000 7f160ffff640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 0x7f16040a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.344+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f16080cf910 con 0x7f1618102d40 2024-09-19T05:54:33.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.344+0000 7f160ffff640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 0x7f16040a5780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f1600005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:33.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.348+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1608009de0 con 0x7f1618102d40 2024-09-19T05:54:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.561+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1618106c50 con 0x7f1618102d40 2024-09-19T05:54:33.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.562+0000 7f160f7fe640 1 -- 172.21.15.38:0/3496907456 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f160809c270 con 0x7f1618102d40 2024-09-19T05:54:33.564 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:54:33.564 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:54:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:54:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 msgr2=0x7f16040a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 0x7f16040a5780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f1600005a20 comp rx=0 tx=0).stop 2024-09-19T05:54:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 msgr2=0x7f16181009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f160800d950 tx=0x7f160800de10 comp rx=0 tx=0).stop 2024-09-19T05:54:33.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 shutdown_connections 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f16040a32c0 0x7f16040a5780 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f161810c170 0x7f1618101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1618103660 0x7f1618100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 --2- 172.21.15.38:0/3496907456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1618102d40 0x7f16181009a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 >> 172.21.15.38:0/3496907456 conn(0x7f16180fc7a0 msgr2=0x7f16180fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.564+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 shutdown_connections 2024-09-19T05:54:33.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:33.565+0000 7f161ce87640 1 -- 172.21.15.38:0/3496907456 wait complete. 2024-09-19T05:54:33.880 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:33 smithi038 ceph-mon[31227]: pgmap v752: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:33.880 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:33 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3496907456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:33 smithi179 ceph-mon[35781]: pgmap v752: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:33 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3496907456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:33.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:33 smithi047 ceph-mon[35327]: pgmap v752: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:33.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:33 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3496907456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:34.239 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T05:54:34.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T05:54:34.503 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/4227173210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8106f10 msgr2=0x7f8cf810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/4227173210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8106f10 0x7f8cf810ea90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009ca0 tx=0x7f8cf002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/4227173210 shutdown_connections 2024-09-19T05:54:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/4227173210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8106f10 0x7f8cf810ea90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/4227173210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8102790 0x7f8cf81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/4227173210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 0x7f8cf81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/4227173210 >> 172.21.15.38:0/4227173210 conn(0x7f8cf80fd8c0 msgr2=0x7f8cf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/4227173210 shutdown_connections 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/4227173210 wait complete. 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.101+0000 7f8cfe2ad640 1 Processor -- start 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 -- start start 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 0x7f8cf80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8106f10 0x7f8cf8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8111f70 con 0x7f8cf8102790 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8111df0 con 0x7f8cf8101de0 2024-09-19T05:54:35.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfe2ad640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81120f0 con 0x7f8cf8106f10 2024-09-19T05:54:35.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36582/0 (socket says 172.21.15.38:36582) 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 -- 172.21.15.38:0/486571914 learned_addr learned my addr 172.21.15.38:0/486571914 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 0x7f8cf80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8106f10 0x7f8cf8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 -- 172.21.15.38:0/486571914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8106f10 msgr2=0x7f8cf8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8106f10 0x7f8cf8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 -- 172.21.15.38:0/486571914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 msgr2=0x7f8cf80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 0x7f8cf80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.102+0000 7f8ceffff640 1 -- 172.21.15.38:0/486571914 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81a6930 con 0x7f8cf8102790 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8ceffff640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000d770 tx=0x7f8ce000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8cf5ffb640 1 -- 172.21.15.38:0/486571914 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0017070 con 0x7f8cf8102790 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf81a6c20 con 0x7f8cf8102790 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8cf5ffb640 1 -- 172.21.15.38:0/486571914 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce0005500 con 0x7f8cf8102790 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8cf5ffb640 1 -- 172.21.15.38:0/486571914 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0004e00 con 0x7f8cf8102790 2024-09-19T05:54:35.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.103+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a7160 con 0x7f8cf8102790 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.105+0000 7f8cf5ffb640 1 -- 172.21.15.38:0/486571914 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8ce0004fa0 con 0x7f8cf8102790 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 Processor -- start 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 -- start start 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557927446940 0x557927448d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5579273a12a0 con 0x55792743c080 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557927348b10 con 0x55792743b760 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8d00686740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55792739d260 con 0x557927446940 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8cf5ffb640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 0x7f8ce40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36596/0 (socket says 172.21.15.38:36596) 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.106+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4120534501 learned_addr learned my addr 172.21.15.38:0/4120534501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 0x7f8ce40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf5ffb640 1 -- 172.21.15.38:0/486571914 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f8ce00cf5c0 con 0x7f8cf8102790 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8ceffff640 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557927446940 0x557927448d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4120534501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557927446940 msgr2=0x557927448d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557927446940 0x557927448d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4120534501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 msgr2=0x55792743bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4120534501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557927112b90 con 0x55792743c080 2024-09-19T05:54:35.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800bd10 tx=0x7f8ce80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e2a866a3b5332e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 0x7f8ce40a5780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0004af0 tx=0x7f8cf002f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cee7fc640 1 -- 172.21.15.38:0/4120534501 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8004030 con 0x55792743c080 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cee7fc640 1 -- 172.21.15.38:0/4120534501 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce80041d0 con 0x55792743c080 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.107+0000 7f8cee7fc640 1 -- 172.21.15.38:0/4120534501 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80117e0 con 0x55792743c080 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- 172.21.15.38:0/4120534501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 msgr2=0x557927446400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800bd10 tx=0x7f8ce80107a0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- 172.21.15.38:0/4120534501 shutdown_connections 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557927446940 0x557927448d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743c080 0x557927446400 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120534501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- 172.21.15.38:0/4120534501 >> 172.21.15.38:0/4120534501 conn(0x557927437cf0 msgr2=0x5579274380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- 172.21.15.38:0/4120534501 shutdown_connections 2024-09-19T05:54:35.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- 172.21.15.38:0/4120534501 wait complete. 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.108+0000 7f8d00686740 1 -- start start 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 Processor -- start 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x557927443d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x5579274442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557927449510 con 0x55792743b760 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5579273296e0 con 0x557927446940 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557927449690 con 0x55792743c080 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8d00686740 10 client.0 _ll_register_callbacks cb 0x55792734da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36602/0 (socket says 172.21.15.38:36602) 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4245660000 learned_addr learned my addr 172.21.15.38:0/4245660000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:35.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8ceffff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x557927443d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.109+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x5579274442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 msgr2=0x557927443d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x557927443d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4245660000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 msgr2=0x5579274442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x5579274442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 -- 172.21.15.38:0/4245660000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557927452a00 con 0x55792743b760 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x5579274442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8010ca0 tx=0x7f8ce8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f25442b32a86916 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8029060 con 0x55792743b760 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f8ce80035f0 con 0x55792743b760 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce80041d0 con 0x55792743b760 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ce8030050 con 0x55792743b760 2024-09-19T05:54:35.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8d00686740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:35.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557927455390 con 0x55792743b760 2024-09-19T05:54:35.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.110+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce80041d0 con 0x55792743b760 2024-09-19T05:54:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8ce80046c0 con 0x55792743b760 2024-09-19T05:54:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 10 client.15978 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 20 client.15978 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:35.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x557927456850 0x557927458c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 4 client.15978 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579273ae9d0 con 0x557927456850 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8ceffff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x557927456850 0x557927458c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8ceffff640 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x557927456850 0x557927458c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da32c65c8e321c8b server_cookie=dd804384a9a40724 in_seq=0 out_seq=0 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.111+0000 7f8cd77fe640 10 client.15978 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4245660000 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579273ae9d0 con 0x557927456850 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8cd77fe640 10 client.15978 handle_command_reply: tid=0 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 client.15978 shutdown 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 msgr2=0x557927443820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8010ca0 tx=0x7f8ce8003f70 comp rx=0 tx=0).stop 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 shutdown_connections 2024-09-19T05:54:35.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x557927456850 0x557927458c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x5579274442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x557927443d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 --2- 172.21.15.38:0/4245660000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x557927443820 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 >> 172.21.15.38:0/4245660000 conn(0x557927437cf0 msgr2=0x55792743da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.112+0000 7f8cd77fe640 0 client.15978 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.113+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 shutdown_connections 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.113+0000 7f8d00686740 1 -- 172.21.15.38:0/4245660000 wait complete. 2024-09-19T05:54:35.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.113+0000 7f8d00686740 20 client.15978 trim_cache size 0 max 16384 2024-09-19T05:54:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 Processor -- start 2024-09-19T05:54:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 -- start start 2024-09-19T05:54:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792745e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b0eb0 con 0x557927446940 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b29f0 con 0x55792743b760 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8d00686740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1580 con 0x55792743c080 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36604/0 (socket says 172.21.15.38:36604) 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 -- 172.21.15.38:0/4120819746 learned_addr learned my addr 172.21.15.38:0/4120819746 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8ceffff640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792745e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 -- 172.21.15.38:0/4120819746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 msgr2=0x55792745e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792745e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 -- 172.21.15.38:0/4120819746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 msgr2=0x55792743e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 -- 172.21.15.38:0/4120819746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557927437b20 con 0x557927446940 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.130+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cf002ade0 tx=0x7f8cf0004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a135e7382ee930bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8cedffb640 1 -- 172.21.15.38:0/4120819746 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0003e00 con 0x557927446940 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8cedffb640 1 -- 172.21.15.38:0/4120819746 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf0004520 con 0x557927446940 2024-09-19T05:54:35.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8cedffb640 1 -- 172.21.15.38:0/4120819746 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf002fcf0 con 0x557927446940 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 -- 172.21.15.38:0/4120819746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 msgr2=0x55792745e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cf002ade0 tx=0x7f8cf0004a80 comp rx=0 tx=0).stop 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 -- 172.21.15.38:0/4120819746 shutdown_connections 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557927446940 0x55792745e750 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792745e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 --2- 172.21.15.38:0/4120819746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55792743b760 0x55792743e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 -- 172.21.15.38:0/4120819746 >> 172.21.15.38:0/4120819746 conn(0x557927437cf0 msgr2=0x5579274420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 -- 172.21.15.38:0/4120819746 shutdown_connections 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.131+0000 7f8d00686740 1 -- 172.21.15.38:0/4120819746 wait complete. 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 -- start start 2024-09-19T05:54:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 Processor -- start 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792744ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x55792744b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b2320 con 0x55792743b760 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1c50 con 0x557927446940 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81aee90 con 0x55792743c080 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8d00686740 10 client.0 _ll_register_callbacks cb 0x7f8ce800b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36610/0 (socket says 172.21.15.38:36610) 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8cf7fff640 1 -- 172.21.15.38:0/1325360386 learned_addr learned my addr 172.21.15.38:0/1325360386 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.132+0000 7f8ceffff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792744ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x55792744b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 -- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 msgr2=0x55792744ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792744ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 -- 172.21.15.38:0/1325360386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 msgr2=0x55792744b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x55792744b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 -- 172.21.15.38:0/1325360386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557927433260 con 0x55792743b760 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cfc84a640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x55792744b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cf7fff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80107e0 tx=0x7f8ce8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48567979295d25d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8039060 con 0x55792743b760 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f8ce8004590 con 0x55792743b760 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf0004520 con 0x55792743b760 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ce8040050 con 0x55792743b760 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8d00686740 10 client.15984 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:35.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579274c1570 con 0x55792743b760 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.133+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf0004520 con 0x55792743b760 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8ce8004f40 con 0x55792743b760 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 10 client.15984 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 20 client.15984 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5579274c1ca0 0x5579274c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 4 client.15984 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5579273a8d20 con 0x5579274c1ca0 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8ceffff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5579274c1ca0 0x5579274c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8ceffff640 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5579274c1ca0 0x5579274c4090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6147fe23f5dd4cca server_cookie=1e1e031e1045e2dd in_seq=0 out_seq=0 2024-09-19T05:54:35.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.134+0000 7f8cd77fe640 10 client.15984 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.144+0000 7f8cd77fe640 1 -- 172.21.15.38:0/1325360386 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579273a8d20 con 0x5579274c1ca0 2024-09-19T05:54:35.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.144+0000 7f8cd77fe640 10 client.15984 handle_command_reply: tid=0 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.144+0000 7f8d00686740 1 client.15984 shutdown 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.144+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 msgr2=0x55792744a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.144+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80107e0 tx=0x7f8ce8005850 comp rx=0 tx=0).stop 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 shutdown_connections 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5579274c1ca0 0x5579274c4090 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557927446940 0x55792744b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55792743c080 0x55792744ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 --2- 172.21.15.38:0/1325360386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55792743b760 0x55792744a920 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 >> 172.21.15.38:0/1325360386 conn(0x557927437cf0 msgr2=0x557927436650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cd77fe640 0 client.15984 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 shutdown_connections 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 1 -- 172.21.15.38:0/1325360386 wait complete. 2024-09-19T05:54:35.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8d00686740 20 client.15984 trim_cache size 0 max 16384 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 46, 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "3688a1e7": { 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 46, 2024-09-19T05:54:35.148 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 142.2, 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:35.149 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:54:35.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 msgr2=0x7f8ce40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 0x7f8ce40a5780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0004af0 tx=0x7f8cf002f310 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 msgr2=0x7f8cf8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000d770 tx=0x7f8ce000dc30 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 shutdown_connections 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8ce40a32c0 0x7f8ce40a5780 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cf8106f10 0x7f8cf8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cf8102790 0x7f8cf8074a30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 --2- 172.21.15.38:0/486571914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8cf8101de0 0x7f8cf80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.145+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 >> 172.21.15.38:0/486571914 conn(0x7f8cf80fd8c0 msgr2=0x7f8cf80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.146+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 shutdown_connections 2024-09-19T05:54:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:35.146+0000 7f8cfe2ad640 1 -- 172.21.15.38:0/486571914 wait complete. 2024-09-19T05:54:35.866 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:35 smithi038 ceph-mon[31227]: pgmap v753: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-19T05:54:35.903 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3688a1e7', quiesce took 3.4 seconds. Will release after: 18.8 2024-09-19T05:54:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:35 smithi179 ceph-mon[35781]: pgmap v753: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-19T05:54:35.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:35 smithi047 ceph-mon[35327]: pgmap v753: 129 pgs: 129 active+clean; 6.0 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-19T05:54:37.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:37 smithi179 ceph-mon[35781]: pgmap v754: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-19T05:54:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:37 smithi047 ceph-mon[35327]: pgmap v754: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-19T05:54:38.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:37 smithi038 ceph-mon[31227]: pgmap v754: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-19T05:54:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:39 smithi179 ceph-mon[35781]: pgmap v755: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:54:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:39 smithi047 ceph-mon[35327]: pgmap v755: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:54:40.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:39 smithi038 ceph-mon[31227]: pgmap v755: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-09-19T05:54:41.947 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:41 smithi038 ceph-mon[31227]: pgmap v756: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:41 smithi047 ceph-mon[35327]: pgmap v756: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:42.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:41 smithi179 ceph-mon[35781]: pgmap v756: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:42.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:42 smithi047 ceph-mon[35327]: pgmap v757: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:43.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:42 smithi038 ceph-mon[31227]: pgmap v757: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:43.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:42 smithi179 ceph-mon[35781]: pgmap v757: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:54:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:44 smithi179 ceph-mon[35781]: pgmap v758: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:54:45.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:44 smithi047 ceph-mon[35327]: pgmap v758: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:54:45.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:44 smithi038 ceph-mon[31227]: pgmap v758: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:54:47.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:46 smithi179 ceph-mon[35781]: pgmap v759: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T05:54:47.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:46 smithi047 ceph-mon[35327]: pgmap v759: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T05:54:47.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:46 smithi038 ceph-mon[31227]: pgmap v759: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-19T05:54:49.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:48 smithi047 ceph-mon[35327]: pgmap v760: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-19T05:54:49.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:48 smithi038 ceph-mon[31227]: pgmap v760: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-19T05:54:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:48 smithi179 ceph-mon[35781]: pgmap v760: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-19T05:54:51.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:50 smithi047 ceph-mon[35327]: pgmap v761: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:54:51.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:50 smithi038 ceph-mon[31227]: pgmap v761: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:54:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:50 smithi179 ceph-mon[35781]: pgmap v761: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-19T05:54:53.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:52 smithi038 ceph-mon[31227]: pgmap v762: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:54:53.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:52 smithi047 ceph-mon[35327]: pgmap v762: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:54:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:52 smithi179 ceph-mon[35781]: pgmap v762: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:54:54.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:54:54.979 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:55.101 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:54 smithi038 ceph-mon[31227]: pgmap v763: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:55.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:54 smithi047 ceph-mon[35327]: pgmap v763: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:55.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:54 smithi179 ceph-mon[35781]: pgmap v763: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:54:55.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.500+0000 7fb1685c5640 1 -- 172.21.15.38:0/2344640844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160102790 msgr2=0x7fb1601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.500+0000 7fb1685c5640 1 --2- 172.21.15.38:0/2344640844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160102790 0x7fb1601067e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb150009900 tx=0x7fb15002a410 comp rx=0 tx=0).stop 2024-09-19T05:54:55.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 -- 172.21.15.38:0/2344640844 shutdown_connections 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 --2- 172.21.15.38:0/2344640844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160106f10 0x7fb16010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 --2- 172.21.15.38:0/2344640844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160102790 0x7fb1601067e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 --2- 172.21.15.38:0/2344640844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160101de0 0x7fb1601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 -- 172.21.15.38:0/2344640844 >> 172.21.15.38:0/2344640844 conn(0x7fb1600fd8c0 msgr2=0x7fb1600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 -- 172.21.15.38:0/2344640844 shutdown_connections 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 -- 172.21.15.38:0/2344640844 wait complete. 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 Processor -- start 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.501+0000 7fb1685c5640 1 -- start start 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160102790 0x7fb1601989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160106f10 0x7fb16019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb160111f90 con 0x7fb160101de0 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb160111e10 con 0x7fb160102790 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb160112110 con 0x7fb160106f10 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49652/0 (socket says 172.21.15.38:49652) 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 -- 172.21.15.38:0/1779453715 learned_addr learned my addr 172.21.15.38:0/1779453715 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb165b60640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160102790 0x7fb1601989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166b62640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160106f10 0x7fb16019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 -- 172.21.15.38:0/1779453715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160106f10 msgr2=0x7fb16019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160106f10 0x7fb16019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 -- 172.21.15.38:0/1779453715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160102790 msgr2=0x7fb1601989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160102790 0x7fb1601989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 -- 172.21.15.38:0/1779453715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16019d440 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb166361640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b4a0 tx=0x7fb15c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c014070 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.502+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16019d730 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.503+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c010af0 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.503+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c017d10 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.503+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1601a5010 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.503+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1601021c0 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.505+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb15c017070 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.505+0000 7fb1577fe640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 0x7fb14c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.505+0000 7fb165b60640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 0x7fb14c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.506+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb15c0cf510 con 0x7fb160101de0 2024-09-19T05:54:55.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.506+0000 7fb165b60640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 0x7fb14c0a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb150009230 tx=0x7fb1500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:55.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.508+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb15c09be10 con 0x7fb160101de0 2024-09-19T05:54:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.720+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1601997f0 con 0x7fb160101de0 2024-09-19T05:54:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.721+0000 7fb1577fe640 1 -- 172.21.15.38:0/1779453715 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fb1601997f0 con 0x7fb160101de0 2024-09-19T05:54:55.724 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:54:55.724 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:54:55.724 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 msgr2=0x7fb14c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 0x7fb14c0a5700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb150009230 tx=0x7fb1500058d0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 msgr2=0x7fb160198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b4a0 tx=0x7fb15c00b960 comp rx=0 tx=0).stop 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 shutdown_connections 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb14c0a3240 0x7fb14c0a5700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb160106f10 0x7fb16019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb160102790 0x7fb1601989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 --2- 172.21.15.38:0/1779453715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb160101de0 0x7fb160198490 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:55.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.723+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 >> 172.21.15.38:0/1779453715 conn(0x7fb1600fd8c0 msgr2=0x7fb1600fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:55.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.724+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 shutdown_connections 2024-09-19T05:54:55.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:55.724+0000 7fb1685c5640 1 -- 172.21.15.38:0/1779453715 wait complete. 2024-09-19T05:54:56.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:55 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1779453715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:56.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:55 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1779453715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:56.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:55 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1779453715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:54:56.731 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 3688a1e7 --release --await' 2024-09-19T05:54:56.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 3688a1e7 --release --await 2024-09-19T05:54:57.000 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:54:57.226 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:56 smithi047 ceph-mon[35327]: pgmap v764: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:54:57.250 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:56 smithi038 ceph-mon[31227]: pgmap v764: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:54:57.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:56 smithi179 ceph-mon[35781]: pgmap v764: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:54:57.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 -- 172.21.15.38:0/2363019635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 msgr2=0x7f48cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/2363019635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc106e60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009930 tx=0x7f48bc02a420 comp rx=0 tx=0).stop 2024-09-19T05:54:57.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 -- 172.21.15.38:0/2363019635 shutdown_connections 2024-09-19T05:54:57.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/2363019635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc101420 0x7f48cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/2363019635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc100a80 0x7f48cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/2363019635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc106e60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.502+0000 7f48d1d2a640 1 -- 172.21.15.38:0/2363019635 >> 172.21.15.38:0/2363019635 conn(0x7f48cc0fc7a0 msgr2=0x7f48cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.503+0000 7f48d1d2a640 1 -- 172.21.15.38:0/2363019635 shutdown_connections 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.503+0000 7f48d1d2a640 1 -- 172.21.15.38:0/2363019635 wait complete. 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.503+0000 7f48d1d2a640 1 Processor -- start 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.503+0000 7f48d1d2a640 1 -- start start 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc100a80 0x7f48cc1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc101420 0x7f48cc1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc110e40 con 0x7f48cc106a80 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc110cc0 con 0x7f48cc101420 2024-09-19T05:54:57.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48d1d2a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc110fc0 con 0x7f48cc100a80 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59224/0 (socket says 172.21.15.38:59224) 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 -- 172.21.15.38:0/265714350 learned_addr learned my addr 172.21.15.38:0/265714350 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc100a80 0x7f48cc1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48caffd640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc101420 0x7f48cc1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 -- 172.21.15.38:0/265714350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc100a80 msgr2=0x7f48cc1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc100a80 0x7f48cc1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 -- 172.21.15.38:0/265714350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc101420 msgr2=0x7f48cc1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc101420 0x7f48cc1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 -- 172.21.15.38:0/265714350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc19f230 con 0x7f48cc106a80 2024-09-19T05:54:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.504+0000 7f48cbfff640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f48c400eaa0 tx=0x7f48c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.505+0000 7f48c8ff9640 1 -- 172.21.15.38:0/265714350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400cdb0 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.505+0000 7f48c8ff9640 1 -- 172.21.15.38:0/265714350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c4005020 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.505+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc19f520 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.505+0000 7f48c8ff9640 1 -- 172.21.15.38:0/265714350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40116b0 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.505+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1ab590 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.507+0000 7f48c8ff9640 1 -- 172.21.15.38:0/265714350 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f48c4005560 con 0x7f48cc106a80 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.507+0000 7f48d4103740 1 Processor -- start 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 -- start start 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66980 0x555706e70d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555706dcb620 con 0x555706e71240 2024-09-19T05:54:57.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555706d72e50 con 0x555706e66980 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48d4103740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555706dc75e0 con 0x555706e66060 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59238/0 (socket says 172.21.15.38:59238) 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 -- 172.21.15.38:0/2767561999 learned_addr learned my addr 172.21.15.38:0/2767561999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48c8ff9640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 0x7f48b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66980 0x555706e70d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48cbfff640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 0x7f48b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 -- 172.21.15.38:0/2767561999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 msgr2=0x555706e66440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e66440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 -- 172.21.15.38:0/2767561999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66980 msgr2=0x555706e70d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66980 0x555706e70d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48caffd640 1 -- 172.21.15.38:0/2767561999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555706d63d40 con 0x555706e71240 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.508+0000 7f48c8ff9640 1 -- 172.21.15.38:0/265714350 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f48c40cff20 con 0x7f48cc106a80 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48caffd640 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005fd0 tx=0x7f48c000b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7231783255c5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48b1ffb640 1 -- 172.21.15.38:0/2767561999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0013270 con 0x555706e71240 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48cbfff640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 0x7f48b40a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f48c4098580 tx=0x7f48c406ae60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48b1ffb640 1 -- 172.21.15.38:0/2767561999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c000ec30 con 0x555706e71240 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48b1ffb640 1 -- 172.21.15.38:0/2767561999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0011b30 con 0x555706e71240 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48d4103740 1 -- 172.21.15.38:0/2767561999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 msgr2=0x555706e73630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.509+0000 7f48d4103740 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005fd0 tx=0x7f48c000b440 comp rx=0 tx=0).stop 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- 172.21.15.38:0/2767561999 shutdown_connections 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e73630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66980 0x555706e70d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- 172.21.15.38:0/2767561999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e66440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- 172.21.15.38:0/2767561999 >> 172.21.15.38:0/2767561999 conn(0x555706e625f0 msgr2=0x555706e629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- 172.21.15.38:0/2767561999 shutdown_connections 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- 172.21.15.38:0/2767561999 wait complete. 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- start start 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 Processor -- start 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e6e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x555706e73e10 con 0x555706e66060 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555706d53f10 con 0x555706e71240 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555706e73f90 con 0x555706e66980 2024-09-19T05:54:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.510+0000 7f48d4103740 10 client.0 _ll_register_callbacks cb 0x555706d77db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59252/0 (socket says 172.21.15.38:59252) 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 -- 172.21.15.38:0/3993117152 learned_addr learned my addr 172.21.15.38:0/3993117152 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48cb7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e6e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 -- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 msgr2=0x555706e6e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e6e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 -- 172.21.15.38:0/3993117152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 msgr2=0x555706e6ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e6ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 -- 172.21.15.38:0/3993117152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555706ada580 con 0x555706e66060 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48caffd640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005e10 tx=0x7f48c0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4873bf202c43f02a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0011540 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f48c000e920 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c000ec30 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48c0005020 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48d4103740 10 client.16002 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.511+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555706ba4960 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c000ec30 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f48c00046c0 con 0x555706e66060 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 10 client.16002 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 20 client.16002 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706e80ca0 0x555706e83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 4 client.16002 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555706dd8b90 con 0x555706e80ca0 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.512+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706e80ca0 0x555706e83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706e80ca0 0x555706e83090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d11c79240457062 server_cookie=a473630c150c7b58 in_seq=0 out_seq=0 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48aaffd640 10 client.16002 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48aaffd640 1 -- 172.21.15.38:0/3993117152 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555706dd8b90 con 0x555706e80ca0 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48aaffd640 10 client.16002 handle_command_reply: tid=0 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 client.16002 shutdown 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 msgr2=0x555706e6e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f48c0005e10 tx=0x7f48c0001e70 comp rx=0 tx=0).stop 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 shutdown_connections 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706e80ca0 0x555706e83090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e6ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.513+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e6e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48d4103740 1 --2- 172.21.15.38:0/3993117152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66060 0x555706e6e190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 >> 172.21.15.38:0/3993117152 conn(0x555706e625f0 msgr2=0x555706e682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48aaffd640 0 client.16002 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 shutdown_connections 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48d4103740 1 -- 172.21.15.38:0/3993117152 wait complete. 2024-09-19T05:54:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.514+0000 7f48d4103740 20 client.16002 trim_cache size 0 max 16384 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 Processor -- start 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 -- start start 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b5330 con 0x555706e66980 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b6e70 con 0x555706e71240 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48d4103740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b5a00 con 0x555706e66060 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59254/0 (socket says 172.21.15.38:59254) 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48cb7fe640 1 -- 172.21.15.38:0/2157830973 learned_addr learned my addr 172.21.15.38:0/2157830973 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48caffd640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.531+0000 7f48caffd640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 -- 172.21.15.38:0/2157830973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 msgr2=0x555706e681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 -- 172.21.15.38:0/2157830973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 msgr2=0x555706e865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 -- 172.21.15.38:0/2157830973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555706e73f20 con 0x555706e66980 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48caffd640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f48bc000c00 tx=0x7f48bc031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d9007e1dfc4c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48b17fa640 1 -- 172.21.15.38:0/2157830973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc04d070 con 0x555706e66980 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48b17fa640 1 -- 172.21.15.38:0/2157830973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc005500 con 0x555706e66980 2024-09-19T05:54:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.532+0000 7f48b17fa640 1 -- 172.21.15.38:0/2157830973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc035040 con 0x555706e66980 2024-09-19T05:54:57.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- 172.21.15.38:0/2157830973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 msgr2=0x555706e860b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f48bc000c00 tx=0x7f48bc031b20 comp rx=0 tx=0).stop 2024-09-19T05:54:57.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- 172.21.15.38:0/2157830973 shutdown_connections 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e71240 0x555706e865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e66980 0x555706e860b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- 172.21.15.38:0/2157830973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66060 0x555706e681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- 172.21.15.38:0/2157830973 >> 172.21.15.38:0/2157830973 conn(0x555706e625f0 msgr2=0x555706e67ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- 172.21.15.38:0/2157830973 shutdown_connections 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- 172.21.15.38:0/2157830973 wait complete. 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 -- start start 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 Processor -- start 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66060 0x555706e83dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.533+0000 7f48d4103740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e84300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b67a0 con 0x555706e71240 2024-09-19T05:54:57.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b60d0 con 0x555706e66060 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b3310 con 0x555706e66980 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 10 client.0 _ll_register_callbacks cb 0x7f48c0013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59268/0 (socket says 172.21.15.38:59268) 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 -- 172.21.15.38:0/2281604980 learned_addr learned my addr 172.21.15.38:0/2281604980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66060 0x555706e83dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e84300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 -- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 msgr2=0x555706e84300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e84300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 -- 172.21.15.38:0/2281604980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66060 msgr2=0x555706e83dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66060 0x555706e83dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 -- 172.21.15.38:0/2281604980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555706eea5e0 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48caffd640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f48c0013fd0 tx=0x7f48c0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d546f1dcf452a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c000e920 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f48c000eac0 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.534+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc005500 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48c00051c0 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555706eeb880 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc005500 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f48c0011a20 con 0x555706e71240 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 10 client.16008 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 20 client.16008 populate_metadata read hostname 'smithi038' 2024-09-19T05:54:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706eebe80 0x555706eee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:54:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 4 client.16008 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3688a1e7", "release": true, "await": true}] 2024-09-19T05:54:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3688a1e7", "release": true, "await": true}) -- 0x555706dd30a0 con 0x555706eebe80 2024-09-19T05:54:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.535+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706eebe80 0x555706eee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:54:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.536+0000 7f48aaffd640 10 client.16008 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:57.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.536+0000 7f48cb7fe640 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706eebe80 0x555706eee270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556030456682e2c3 server_cookie=b9103de0ce5ed10f in_seq=0 out_seq=0 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48aaffd640 1 -- 172.21.15.38:0/2281604980 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555706dd30a0 con 0x555706eebe80 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48aaffd640 10 client.16008 handle_command_reply: tid=0 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 client.16008 shutdown 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 msgr2=0x555706e84840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f48c0013fd0 tx=0x7f48c0001e70 comp rx=0 tx=0).stop 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 shutdown_connections 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x555706eebe80 0x555706eee270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555706e71240 0x555706e84840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555706e66980 0x555706e84300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 --2- 172.21.15.38:0/2281604980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555706e66060 0x555706e83dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 >> 172.21.15.38:0/2281604980 conn(0x555706e625f0 msgr2=0x555706e61750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48aaffd640 0 client.16008 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 shutdown_connections 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 1 -- 172.21.15.38:0/2281604980 wait complete. 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.539+0000 7f48d4103740 20 client.16008 trim_cache size 0 max 16384 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 48, 2024-09-19T05:54:57.545 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "3688a1e7": { 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 48, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 142.2, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:54:57.546 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:54:57.547 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 msgr2=0x7f48b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 0x7f48b40a5700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f48c4098580 tx=0x7f48c406ae60 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 msgr2=0x7f48cc19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f48c400eaa0 tx=0x7f48c400ef60 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 shutdown_connections 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f48b40a3240 0x7f48b40a5700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48cc106a80 0x7f48cc19eb00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48cc101420 0x7f48cc1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 --2- 172.21.15.38:0/265714350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48cc100a80 0x7f48cc1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 >> 172.21.15.38:0/265714350 conn(0x7f48cc0fc7a0 msgr2=0x7f48cc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:54:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 shutdown_connections 2024-09-19T05:54:57.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:54:57.540+0000 7f48d1d2a640 1 -- 172.21.15.38:0/265714350 wait complete. 2024-09-19T05:54:58.390 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3688a1e7', total seconds elapsed: 25.8 2024-09-19T05:54:59.249 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:54:58 smithi179 ceph-mon[35781]: pgmap v765: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:54:59.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:54:58 smithi038 ceph-mon[31227]: pgmap v765: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:54:59.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:54:58 smithi047 ceph-mon[35327]: pgmap v765: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:55:01.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:00 smithi038 ceph-mon[31227]: pgmap v766: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:55:01.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:00 smithi179 ceph-mon[35781]: pgmap v766: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:55:01.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:00 smithi047 ceph-mon[35327]: pgmap v766: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:55:03.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:02 smithi038 ceph-mon[31227]: pgmap v767: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:55:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:02 smithi179 ceph-mon[35781]: pgmap v767: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:55:03.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:02 smithi047 ceph-mon[35327]: pgmap v767: 129 pgs: 129 active+clean; 6.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T05:55:05.396 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:04 smithi038 ceph-mon[31227]: pgmap v768: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-19T05:55:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:04 smithi179 ceph-mon[35781]: pgmap v768: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-19T05:55:05.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:04 smithi047 ceph-mon[35327]: pgmap v768: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-19T05:55:07.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:06 smithi038 ceph-mon[31227]: pgmap v769: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:55:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:06 smithi179 ceph-mon[35781]: pgmap v769: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:55:07.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:06 smithi047 ceph-mon[35327]: pgmap v769: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-09-19T05:55:09.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:08 smithi038 ceph-mon[31227]: pgmap v770: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:08 smithi179 ceph-mon[35781]: pgmap v770: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:09.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:08 smithi047 ceph-mon[35327]: pgmap v770: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:11.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:11 smithi038 ceph-mon[31227]: pgmap v771: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:11 smithi179 ceph-mon[35781]: pgmap v771: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:11.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:11 smithi047 ceph-mon[35327]: pgmap v771: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:13.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:13 smithi038 ceph-mon[31227]: pgmap v772: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:13.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:13 smithi179 ceph-mon[35781]: pgmap v772: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:13.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:13 smithi047 ceph-mon[35327]: pgmap v772: 129 pgs: 129 active+clean; 6.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:55:15.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:15 smithi038 ceph-mon[31227]: pgmap v773: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:15 smithi179 ceph-mon[35781]: pgmap v773: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:15.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:15 smithi047 ceph-mon[35327]: pgmap v773: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:17 smithi038 ceph-mon[31227]: pgmap v774: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:17.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:55:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:17 smithi179 ceph-mon[35781]: pgmap v774: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:17.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:55:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:17 smithi047 ceph-mon[35327]: pgmap v774: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:55:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:55:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:55:17.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:55:19.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:19 smithi038 ceph-mon[31227]: pgmap v775: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:55:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:19 smithi179 ceph-mon[35781]: pgmap v775: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:55:19.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:19 smithi047 ceph-mon[35327]: pgmap v775: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:55:21.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:21 smithi038 ceph-mon[31227]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:21 smithi179 ceph-mon[35781]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:21.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:21 smithi047 ceph-mon[35327]: pgmap v776: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:23.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:23 smithi038 ceph-mon[31227]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:23.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:23 smithi179 ceph-mon[35781]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:23.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:23 smithi047 ceph-mon[35327]: pgmap v777: 129 pgs: 129 active+clean; 6.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:25.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:25 smithi038 ceph-mon[31227]: pgmap v778: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:25.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:25 smithi179 ceph-mon[35781]: pgmap v778: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:25.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:25 smithi047 ceph-mon[35327]: pgmap v778: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:27.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:27 smithi038 ceph-mon[31227]: pgmap v779: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:27.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:27 smithi179 ceph-mon[35781]: pgmap v779: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:27.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:27 smithi047 ceph-mon[35327]: pgmap v779: 129 pgs: 129 active+clean; 6.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:29.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:29 smithi038 ceph-mon[31227]: pgmap v780: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:29.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:29 smithi179 ceph-mon[35781]: pgmap v780: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:29.728 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:29 smithi047 ceph-mon[35327]: pgmap v780: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:31.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:31 smithi038 ceph-mon[31227]: pgmap v781: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:31.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:31 smithi179 ceph-mon[35781]: pgmap v781: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:31.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:31 smithi047 ceph-mon[35327]: pgmap v781: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:31.795 INFO:tasks.workunit.client.0.smithi038.stdout:437/600: open fd = 3 2024-09-19T05:55:31.795 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:438/600: open fd = 3 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:439/600: open fd = 3 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:440/600: open fd = 3 2024-09-19T05:55:31.796 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:441/600: open fd = 3 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:442/600: open fd = 3 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:443/600: open fd = 3 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.797 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:444/600: open fd = 3 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:445/600: open fd = 3 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:446/600: open fd = 3 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.798 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:447/600: open fd = 3 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:448/600: open fd = 3 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:449/600: open fd = 3 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.799 INFO:tasks.workunit.client.0.smithi038.stdout:450/600: open fd = 3 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:451/600: open fd = 3 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:452/600: open fd = 3 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:453/600: open fd = 3 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.800 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:454/600: open fd = 3 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:455/600: open fd = 3 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:456/600: open fd = 3 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.801 INFO:tasks.workunit.client.0.smithi038.stdout:457/600: open fd = 3 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:458/600: open fd = 3 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:459/600: open fd = 3 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:460/600: open fd = 3 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.802 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:461/600: open fd = 3 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:462/600: open fd = 3 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:463/600: open fd = 3 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.803 INFO:tasks.workunit.client.0.smithi038.stdout:464/600: open fd = 3 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:465/600: open fd = 3 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:466/600: open fd = 3 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:467/600: open fd = 3 2024-09-19T05:55:31.804 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:468/600: open fd = 3 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:469/600: open fd = 3 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:470/600: open fd = 3 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.805 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:471/600: open fd = 3 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:472/600: open fd = 3 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:473/600: open fd = 3 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:474/600: open fd = 3 2024-09-19T05:55:31.806 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:475/600: open fd = 3 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:476/600: open fd = 3 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:477/600: open fd = 3 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.807 INFO:tasks.workunit.client.0.smithi038.stdout:478/600: open fd = 3 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:479/600: open fd = 3 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:480/600: open fd = 3 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:481/600: open fd = 3 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.808 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:482/600: open fd = 3 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:483/600: open fd = 3 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:484/600: open fd = 3 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.809 INFO:tasks.workunit.client.0.smithi038.stdout:485/600: open fd = 3 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:486/600: open fd = 3 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:487/600: open fd = 3 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:488/600: open fd = 3 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.810 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:489/600: open fd = 3 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:490/600: open fd = 3 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:491/600: open fd = 3 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.811 INFO:tasks.workunit.client.0.smithi038.stdout:492/600: open fd = 3 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:493/600: open fd = 3 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.812 INFO:tasks.workunit.client.0.smithi038.stdout:494/600: open fd = 3 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:495/600: open fd = 3 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:496/600: open fd = 3 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.813 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:497/600: open fd = 3 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:498/600: open fd = 3 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:55:31.814 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:55:33.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:33 smithi038 ceph-mon[31227]: pgmap v782: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:33.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:33 smithi179 ceph-mon[35781]: pgmap v782: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:33.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:33 smithi047 ceph-mon[35327]: pgmap v782: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:35.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:35 smithi038 ceph-mon[31227]: pgmap v783: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:35 smithi179 ceph-mon[35781]: pgmap v783: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:35.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:35 smithi047 ceph-mon[35327]: pgmap v783: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:37.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:37 smithi038 ceph-mon[31227]: pgmap v784: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:37.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:37 smithi179 ceph-mon[35781]: pgmap v784: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:37.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:37 smithi047 ceph-mon[35327]: pgmap v784: 129 pgs: 129 active+clean; 6.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:39.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:39 smithi038 ceph-mon[31227]: pgmap v785: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:39.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:39 smithi179 ceph-mon[35781]: pgmap v785: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:39.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:39 smithi047 ceph-mon[35327]: pgmap v785: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:41.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:41 smithi038 ceph-mon[31227]: pgmap v786: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:41.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:41 smithi179 ceph-mon[35781]: pgmap v786: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:41.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:41 smithi047 ceph-mon[35327]: pgmap v786: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:43.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:43 smithi038 ceph-mon[31227]: pgmap v787: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:43.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:43 smithi179 ceph-mon[35781]: pgmap v787: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:43.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:43 smithi047 ceph-mon[35327]: pgmap v787: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:45.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:45 smithi038 ceph-mon[31227]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:45.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:45 smithi179 ceph-mon[35781]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:45.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:45 smithi047 ceph-mon[35327]: pgmap v788: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:55:47.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:47 smithi038 ceph-mon[31227]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:47.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:47 smithi179 ceph-mon[35781]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:47.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:47 smithi047 ceph-mon[35327]: pgmap v789: 129 pgs: 129 active+clean; 6.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:55:49.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:49 smithi038 ceph-mon[31227]: pgmap v790: 129 pgs: 129 active+clean; 6.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:49.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:49 smithi179 ceph-mon[35781]: pgmap v790: 129 pgs: 129 active+clean; 6.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:49.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:49 smithi047 ceph-mon[35327]: pgmap v790: 129 pgs: 129 active+clean; 6.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:51 smithi038 ceph-mon[31227]: pgmap v791: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:51 smithi179 ceph-mon[35781]: pgmap v791: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:51.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:51 smithi047 ceph-mon[35327]: pgmap v791: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:55:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:53 smithi038 ceph-mon[31227]: pgmap v792: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:53 smithi179 ceph-mon[35781]: pgmap v792: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:53.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:53 smithi047 ceph-mon[35327]: pgmap v792: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:55:55.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:55 smithi038 ceph-mon[31227]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:55 smithi179 ceph-mon[35781]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:55.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:55 smithi047 ceph-mon[35327]: pgmap v793: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:55:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:57 smithi038 ceph-mon[31227]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:57 smithi179 ceph-mon[35781]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:57.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:57 smithi047 ceph-mon[35327]: pgmap v794: 129 pgs: 129 active+clean; 6.6 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:55:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:55:59 smithi038 ceph-mon[31227]: pgmap v795: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:55:59 smithi179 ceph-mon[35781]: pgmap v795: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:55:59.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:55:59 smithi047 ceph-mon[35327]: pgmap v795: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:01 smithi179 ceph-mon[35781]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:01.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:01 smithi047 ceph-mon[35327]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:01 smithi038 ceph-mon[31227]: pgmap v796: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:03 smithi179 ceph-mon[35781]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:03 smithi047 ceph-mon[35327]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:03.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:03 smithi038 ceph-mon[31227]: pgmap v797: 129 pgs: 129 active+clean; 6.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:05 smithi179 ceph-mon[35781]: pgmap v798: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:05.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:05 smithi047 ceph-mon[35327]: pgmap v798: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:05.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:05 smithi038 ceph-mon[31227]: pgmap v798: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:07 smithi179 ceph-mon[35781]: pgmap v799: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:07 smithi047 ceph-mon[35327]: pgmap v799: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:07 smithi038 ceph-mon[31227]: pgmap v799: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:09 smithi179 ceph-mon[35781]: pgmap v800: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T05:56:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:09 smithi047 ceph-mon[35327]: pgmap v800: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T05:56:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:09 smithi038 ceph-mon[31227]: pgmap v800: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-09-19T05:56:11.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:11 smithi038 ceph-mon[31227]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:11 smithi179 ceph-mon[35781]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:11 smithi047 ceph-mon[35327]: pgmap v801: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:13 smithi179 ceph-mon[35781]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:13.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:13 smithi047 ceph-mon[35327]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:13.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:13 smithi038 ceph-mon[31227]: pgmap v802: 129 pgs: 129 active+clean; 6.8 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:15 smithi179 ceph-mon[35781]: pgmap v803: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:56:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:15 smithi047 ceph-mon[35327]: pgmap v803: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:56:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:15 smithi038 ceph-mon[31227]: pgmap v803: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:56:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:17 smithi047 ceph-mon[35327]: pgmap v804: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:56:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:17 smithi038 ceph-mon[31227]: pgmap v804: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:56:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:17 smithi179 ceph-mon[35781]: pgmap v804: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:56:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:56:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:56:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:56:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:56:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:19 smithi047 ceph-mon[35327]: pgmap v805: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:19.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:19 smithi038 ceph-mon[31227]: pgmap v805: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:19 smithi179 ceph-mon[35781]: pgmap v805: 129 pgs: 129 active+clean; 6.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:56:20.848 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T05:56:20.848 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T05:56:20.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:56:21.121 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:21.534 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:21 smithi038 ceph-mon[31227]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:21.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 -- 172.21.15.38:0/2758151704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 msgr2=0x7f35381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:21.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 --2- 172.21.15.38:0/2758151704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f35381067e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802a410 comp rx=0 tx=0).stop 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 -- 172.21.15.38:0/2758151704 shutdown_connections 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 --2- 172.21.15.38:0/2758151704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538106f10 0x7f353810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 --2- 172.21.15.38:0/2758151704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f35381067e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 --2- 172.21.15.38:0/2758151704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538101de0 0x7f35381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 -- 172.21.15.38:0/2758151704 >> 172.21.15.38:0/2758151704 conn(0x7f35380fd8c0 msgr2=0x7f35380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 -- 172.21.15.38:0/2758151704 shutdown_connections 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.616+0000 7f353ea89640 1 -- 172.21.15.38:0/2758151704 wait complete. 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 Processor -- start 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 -- start start 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 0x7f353819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:21.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538106f10 0x7f353819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3538111fe0 con 0x7f3538102790 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3538111e60 con 0x7f3538101de0 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353ea89640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3538112160 con 0x7f3538106f10 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f352ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f352ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51372/0 (socket says 172.21.15.38:51372) 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f352ffff640 1 -- 172.21.15.38:0/2373925879 learned_addr learned my addr 172.21.15.38:0/2373925879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.617+0000 7f353c825640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 0x7f353819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f353d026640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538106f10 0x7f353819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 -- 172.21.15.38:0/2373925879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538106f10 msgr2=0x7f353819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:21.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538106f10 0x7f353819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 -- 172.21.15.38:0/2373925879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 msgr2=0x7f353819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 0x7f353819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 -- 172.21.15.38:0/2373925879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353819e6c0 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f353c825640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 0x7f353819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352ffff640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3528004770 tx=0x7f3528003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352803f070 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353819e950 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528031ca0 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528035030 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.618+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381a7320 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.619+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35381021c0 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.620+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3528042070 con 0x7f3538102790 2024-09-19T05:56:21.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.621+0000 7f352dffb640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 0x7f35200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.621+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f35280f0570 con 0x7f3538102790 2024-09-19T05:56:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.621+0000 7f353c825640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 0x7f35200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.621+0000 7f353c825640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 0x7f35200a5700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3534009930 tx=0x7f3534008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.625+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35280bced0 con 0x7f3538102790 2024-09-19T05:56:21.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:21 smithi047 ceph-mon[35327]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:21.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.841+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35381a74f0 con 0x7f3538102790 2024-09-19T05:56:21.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.841+0000 7f352dffb640 1 -- 172.21.15.38:0/2373925879 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f35280bc7d0 con 0x7f3538102790 2024-09-19T05:56:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:56:21.844 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:56:21.844 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 msgr2=0x7f35200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 0x7f35200a5700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3534009930 tx=0x7f3534008040 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 msgr2=0x7f353819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3528004770 tx=0x7f3528003c10 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 shutdown_connections 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f35200a3240 0x7f35200a5700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3538106f10 0x7f353819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3538102790 0x7f353819acb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 --2- 172.21.15.38:0/2373925879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3538101de0 0x7f353819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 >> 172.21.15.38:0/2373925879 conn(0x7f35380fd8c0 msgr2=0x7f35380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:21.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 shutdown_connections 2024-09-19T05:56:21.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:21.844+0000 7f353ea89640 1 -- 172.21.15.38:0/2373925879 wait complete. 2024-09-19T05:56:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:21 smithi179 ceph-mon[35781]: pgmap v806: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:56:22.612 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T05:56:22.613 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T05:56:22.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:22 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2373925879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:22.874 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:22.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:22 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2373925879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:22.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:22 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2373925879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:23.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.353+0000 7fec3d1d2640 1 -- 172.21.15.38:0/749081394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec3810c170 msgr2=0x7fec3810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.353+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/749081394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec3810c170 0x7fec3810e610 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fec30009ca0 tx=0x7fec3002a950 comp rx=0 tx=0).stop 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 -- 172.21.15.38:0/749081394 shutdown_connections 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/749081394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec3810c170 0x7fec3810e610 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/749081394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38103660 0x7fec3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/749081394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec38102d40 0x7fec38103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 -- 172.21.15.38:0/749081394 >> 172.21.15.38:0/749081394 conn(0x7fec380fc7a0 msgr2=0x7fec380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 -- 172.21.15.38:0/749081394 shutdown_connections 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 -- 172.21.15.38:0/749081394 wait complete. 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.354+0000 7fec3d1d2640 1 Processor -- start 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 -- start start 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38102d40 0x7fec381983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 0x7fec3819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec38111f30 con 0x7fec38103660 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec38111db0 con 0x7fec38102d40 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3d1d2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec381120b0 con 0x7fec3810c170 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51402/0 (socket says 172.21.15.38:51402) 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3659c640 1 -- 172.21.15.38:0/3189442983 learned_addr learned my addr 172.21.15.38:0/3189442983 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec36d9d640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38102d40 0x7fec381983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.355+0000 7fec3759e640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 0x7fec3819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 -- 172.21.15.38:0/3189442983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 msgr2=0x7fec3819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 0x7fec3819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 -- 172.21.15.38:0/3189442983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38102d40 msgr2=0x7fec381983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38102d40 0x7fec381983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 -- 172.21.15.38:0/3189442983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3819d380 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3759e640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 0x7fec3819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3659c640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00d740 tx=0x7fec2c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec1ffff640 1 -- 172.21.15.38:0/3189442983 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c017070 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec1ffff640 1 -- 172.21.15.38:0/3189442983 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec2c00de50 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3819d670 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec1ffff640 1 -- 172.21.15.38:0/3189442983 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00f620 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.356+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec381a4f50 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.358+0000 7fec1ffff640 1 -- 172.21.15.38:0/3189442983 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fec2c009c30 con 0x7fec38103660 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.358+0000 7fec3f5ab740 1 Processor -- start 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 -- start start 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55755a21a160 con 0x55755a2b6170 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55755a1c1720 con 0x55755a2c0a30 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55755a216120 con 0x55755a2b5850 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51412/0 (socket says 172.21.15.38:51412) 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 -- 172.21.15.38:0/644925045 learned_addr learned my addr 172.21.15.38:0/644925045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec1ffff640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 0x7fec200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3659c640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3759e640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 -- 172.21.15.38:0/644925045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 msgr2=0x55755a2b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 -- 172.21.15.38:0/644925045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 msgr2=0x55755a2c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec36d9d640 1 -- 172.21.15.38:0/644925045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557559f8bba0 con 0x55755a2b6170 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3759e640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3659c640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec3759e640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 0x7fec200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.359+0000 7fec1ffff640 1 -- 172.21.15.38:0/3189442983 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fec2c0cf840 con 0x7fec38103660 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec36d9d640 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ac20 tx=0x7fec2800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1de4856296259f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec1d7fa640 1 -- 172.21.15.38:0/644925045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2801d070 con 0x55755a2b6170 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec1d7fa640 1 -- 172.21.15.38:0/644925045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28018b20 con 0x55755a2b6170 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec1d7fa640 1 -- 172.21.15.38:0/644925045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28010ba0 con 0x55755a2b6170 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 -- 172.21.15.38:0/644925045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 msgr2=0x55755a2c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ac20 tx=0x7fec2800f790 comp rx=0 tx=0).stop 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3759e640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 0x7fec200a5700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec30030040 tx=0x7fec30004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 -- 172.21.15.38:0/644925045 shutdown_connections 2024-09-19T05:56:23.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b6170 0x55755a2c04f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.360+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/644925045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- 172.21.15.38:0/644925045 >> 172.21.15.38:0/644925045 conn(0x55755a2b1de0 msgr2=0x55755a2b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- 172.21.15.38:0/644925045 shutdown_connections 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- 172.21.15.38:0/644925045 wait complete. 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- start start 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 Processor -- start 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55755a2c3600 con 0x55755a2b5850 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55755a1a2900 con 0x55755a2c0a30 2024-09-19T05:56:23.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55755a2c3780 con 0x55755a2b6170 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec3f5ab740 10 client.0 _ll_register_callbacks cb 0x55755a1a0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51420/0 (socket says 172.21.15.38:51420) 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.361+0000 7fec36d9d640 1 -- 172.21.15.38:0/1852887725 learned_addr learned my addr 172.21.15.38:0/1852887725 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec3759e640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 -- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 msgr2=0x55755a2bde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 -- 172.21.15.38:0/1852887725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 msgr2=0x55755a2be3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2be3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 -- 172.21.15.38:0/1852887725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755a1e2c50 con 0x55755a2b5850 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec3759e640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2be3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec36d9d640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fec28010700 tx=0x7fec280105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1b7c125c7013c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2801d050 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fec28018910 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec28018b20 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec28032050 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec3f5ab740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55755a2cf250 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28018b20 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.362+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fec280046c0 con 0x55755a2b5850 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 10 client.16026 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 20 client.16026 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a2d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 4 client.16026 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:56:23.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755a227890 con 0x55755a2d06b0 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3659c640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a2d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec3659c640 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a2d2aa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f69bb452f7d2f7 server_cookie=32e310a86729c1b4 in_seq=0 out_seq=0 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.363+0000 7fec1aff5640 10 client.16026 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec1aff5640 1 -- 172.21.15.38:0/1852887725 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55755a227890 con 0x55755a2d06b0 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec1aff5640 10 client.16026 handle_command_reply: tid=0 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 client.16026 shutdown 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 msgr2=0x55755a2bd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fec28010700 tx=0x7fec280105b0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 shutdown_connections 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a2d2aa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2be3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec1aff5640 0 client.16026 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:23.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2bde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1852887725 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2bd950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 >> 172.21.15.38:0/1852887725 conn(0x55755a2b1de0 msgr2=0x55755a2b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 shutdown_connections 2024-09-19T05:56:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.364+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1852887725 wait complete. 2024-09-19T05:56:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.365+0000 7fec3f5ab740 20 client.16026 trim_cache size 0 max 16384 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.381+0000 7fec3f5ab740 1 Processor -- start 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.381+0000 7fec3f5ab740 1 -- start start 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2b6170 0x55755a2d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec381aedf0 con 0x55755a2c0a30 2024-09-19T05:56:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec381b0930 con 0x55755a2b6170 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec381af4c0 con 0x55755a2b5850 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51436/0 (socket says 172.21.15.38:51436) 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 -- 172.21.15.38:0/1350530169 learned_addr learned my addr 172.21.15.38:0/1350530169 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3659c640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2b6170 0x55755a2d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec36d9d640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 -- 172.21.15.38:0/1350530169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 msgr2=0x55755a2be130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2be130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 -- 172.21.15.38:0/1350530169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2b6170 msgr2=0x55755a2d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2b6170 0x55755a2d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 -- 172.21.15.38:0/1350530169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755a2ad3c0 con 0x55755a2c0a30 2024-09-19T05:56:23.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec36d9d640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:23.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.382+0000 7fec3759e640 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fec30035000 tx=0x7fec3003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395c545553d8df5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec1cff9640 1 -- 172.21.15.38:0/1350530169 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30034040 con 0x55755a2c0a30 2024-09-19T05:56:23.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec1cff9640 1 -- 172.21.15.38:0/1350530169 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004040 con 0x55755a2c0a30 2024-09-19T05:56:23.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec1cff9640 1 -- 172.21.15.38:0/1350530169 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30044440 con 0x55755a2c0a30 2024-09-19T05:56:23.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1350530169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 msgr2=0x55755a2d8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fec30035000 tx=0x7fec3003f000 comp rx=0 tx=0).stop 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1350530169 shutdown_connections 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2c0a30 0x55755a2d8610 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2b6170 0x55755a2d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/1350530169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b5850 0x55755a2be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1350530169 >> 172.21.15.38:0/1350530169 conn(0x55755a2b1de0 msgr2=0x55755a2bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1350530169 shutdown_connections 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.383+0000 7fec3f5ab740 1 -- 172.21.15.38:0/1350530169 wait complete. 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 -- start start 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 Processor -- start 2024-09-19T05:56:23.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec381b0260 con 0x55755a2b5850 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec381afb90 con 0x55755a2c0a30 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec381acdd0 con 0x55755a2b6170 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3f5ab740 10 client.0 _ll_register_callbacks cb 0x7fec2801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51452/0 (socket says 172.21.15.38:51452) 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 -- 172.21.15.38:0/4134559833 learned_addr learned my addr 172.21.15.38:0/4134559833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3759e640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3659c640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 -- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 msgr2=0x55755a2baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2baea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 -- 172.21.15.38:0/4134559833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 msgr2=0x55755a2bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec36d9d640 1 -- 172.21.15.38:0/4134559833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755a33a0a0 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.384+0000 7fec3759e640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec36d9d640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fec28010700 tx=0x7fec28004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d085bbe535d1e593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2801d040 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fec28010940 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec30004040 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec28046050 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec3f5ab740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55755a33b340 con 0x55755a2b5850 2024-09-19T05:56:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.385+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004040 con 0x55755a2b5850 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fec28004cb0 con 0x55755a2b5850 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 10 client.16032 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 20 client.16032 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a33db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 4 client.16032 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55755a221be0 con 0x55755a2d06b0 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.386+0000 7fec3659c640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a33db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.387+0000 7fec3659c640 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a33db30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbdae4d1f07d31a server_cookie=9148d381434ac881 in_seq=0 out_seq=0 2024-09-19T05:56:23.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.387+0000 7fec1aff5640 10 client.16032 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec1aff5640 1 -- 172.21.15.38:0/4134559833 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55755a221be0 con 0x55755a2d06b0 2024-09-19T05:56:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec1aff5640 10 client.16032 handle_command_reply: tid=0 2024-09-19T05:56:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 client.16032 shutdown 2024-09-19T05:56:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 msgr2=0x55755a2ba960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fec28010700 tx=0x7fec28004990 comp rx=0 tx=0).stop 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 shutdown_connections 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55755a2d06b0 0x55755a33db30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55755a2c0a30 0x55755a2bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55755a2b6170 0x55755a2baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 --2- 172.21.15.38:0/4134559833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55755a2b5850 0x55755a2ba960 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 >> 172.21.15.38:0/4134559833 conn(0x55755a2b1de0 msgr2=0x55755a2b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec1aff5640 0 client.16032 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 shutdown_connections 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 1 -- 172.21.15.38:0/4134559833 wait complete. 2024-09-19T05:56:23.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.397+0000 7fec3f5ab740 20 client.16032 trim_cache size 0 max 16384 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 50, 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "563cc35d": { 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 50, 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:23.401 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.4, 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:23.402 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:56:23.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 msgr2=0x7fec200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 0x7fec200a5700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec30030040 tx=0x7fec30004a40 comp rx=0 tx=0).stop 2024-09-19T05:56:23.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 msgr2=0x7fec38198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00d740 tx=0x7fec2c00dc00 comp rx=0 tx=0).stop 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 shutdown_connections 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fec200a3240 0x7fec200a5700 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3810c170 0x7fec3819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec38103660 0x7fec38198910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 --2- 172.21.15.38:0/3189442983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec38102d40 0x7fec381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 >> 172.21.15.38:0/3189442983 conn(0x7fec380fc7a0 msgr2=0x7fec380fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 shutdown_connections 2024-09-19T05:56:23.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:23.398+0000 7fec3d1d2640 1 -- 172.21.15.38:0/3189442983 wait complete. 2024-09-19T05:56:23.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:23 smithi038 ceph-mon[31227]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:23 smithi179 ceph-mon[35781]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:23.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:23 smithi047 ceph-mon[35327]: pgmap v807: 129 pgs: 129 active+clean; 6.9 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:24.181 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '563cc35d', quiesce took 3.3 seconds. Will release after: 26.15 2024-09-19T05:56:25.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:25 smithi038 ceph-mon[31227]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:56:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:25 smithi179 ceph-mon[35781]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:56:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:25 smithi047 ceph-mon[35327]: pgmap v808: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:56:27.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:27 smithi038 ceph-mon[31227]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:27 smithi179 ceph-mon[35781]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:27.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:27 smithi047 ceph-mon[35327]: pgmap v809: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:56:29.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:29 smithi038 ceph-mon[31227]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-19T05:56:29.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:29 smithi179 ceph-mon[35781]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-19T05:56:29.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:29 smithi047 ceph-mon[35327]: pgmap v810: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-19T05:56:31.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:31 smithi038 ceph-mon[31227]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-19T05:56:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:31 smithi179 ceph-mon[35781]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-19T05:56:31.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:31 smithi047 ceph-mon[35327]: pgmap v811: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-19T05:56:33.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:33 smithi038 ceph-mon[31227]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:33 smithi179 ceph-mon[35781]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:33 smithi047 ceph-mon[35327]: pgmap v812: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:35 smithi038 ceph-mon[31227]: pgmap v813: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:35.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:35 smithi179 ceph-mon[35781]: pgmap v813: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:35.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:35 smithi047 ceph-mon[35327]: pgmap v813: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-09-19T05:56:37.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:37 smithi038 ceph-mon[31227]: pgmap v814: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-19T05:56:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:37 smithi179 ceph-mon[35781]: pgmap v814: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-19T05:56:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:37 smithi047 ceph-mon[35327]: pgmap v814: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-19T05:56:39.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:39 smithi038 ceph-mon[31227]: pgmap v815: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-19T05:56:39.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:39 smithi179 ceph-mon[35781]: pgmap v815: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-19T05:56:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:39 smithi047 ceph-mon[35327]: pgmap v815: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-19T05:56:41.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:41 smithi038 ceph-mon[31227]: pgmap v816: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:56:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:41 smithi179 ceph-mon[35781]: pgmap v816: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:56:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:41 smithi047 ceph-mon[35327]: pgmap v816: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-19T05:56:43.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:43 smithi038 ceph-mon[31227]: pgmap v817: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:43 smithi179 ceph-mon[35781]: pgmap v817: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:43.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:43 smithi047 ceph-mon[35327]: pgmap v817: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:45.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:45 smithi038 ceph-mon[31227]: pgmap v818: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:45 smithi179 ceph-mon[35781]: pgmap v818: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:45 smithi047 ceph-mon[35327]: pgmap v818: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:56:47.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:47 smithi038 ceph-mon[31227]: pgmap v819: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:56:47.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:47 smithi179 ceph-mon[35781]: pgmap v819: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:56:47.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:47 smithi047 ceph-mon[35327]: pgmap v819: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T05:56:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:49 smithi038 ceph-mon[31227]: pgmap v820: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T05:56:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:49 smithi179 ceph-mon[35781]: pgmap v820: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T05:56:49.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:49 smithi047 ceph-mon[35327]: pgmap v820: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T05:56:50.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:56:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 -- 172.21.15.38:0/3658820538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340101de0 msgr2=0x7fd3401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 --2- 172.21.15.38:0/3658820538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340101de0 0x7fd3401021c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd330009900 tx=0x7fd33002a410 comp rx=0 tx=0).stop 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 -- 172.21.15.38:0/3658820538 shutdown_connections 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 --2- 172.21.15.38:0/3658820538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340106f10 0x7fd34010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 --2- 172.21.15.38:0/3658820538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 0x7fd3401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 --2- 172.21.15.38:0/3658820538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340101de0 0x7fd3401021c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.128+0000 7fd346232640 1 -- 172.21.15.38:0/3658820538 >> 172.21.15.38:0/3658820538 conn(0x7fd3400fd8c0 msgr2=0x7fd3400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 -- 172.21.15.38:0/3658820538 shutdown_connections 2024-09-19T05:56:51.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 -- 172.21.15.38:0/3658820538 wait complete. 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 Processor -- start 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 -- start start 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340101de0 0x7fd3401984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 0x7fd3401989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.129+0000 7fd346232640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd340111f70 con 0x7fd340106f10 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd346232640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd340111df0 con 0x7fd340101de0 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd346232640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401120f0 con 0x7fd340102790 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:51.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58864/0 (socket says 172.21.15.38:58864) 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 -- 172.21.15.38:0/2962689191 learned_addr learned my addr 172.21.15.38:0/2962689191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33effd640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 0x7fd3401989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33f7fe640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340101de0 0x7fd3401984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 -- 172.21.15.38:0/2962689191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 msgr2=0x7fd3401989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 0x7fd3401989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 -- 172.21.15.38:0/2962689191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340101de0 msgr2=0x7fd3401984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340101de0 0x7fd3401984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 -- 172.21.15.38:0/2962689191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34019d460 con 0x7fd340106f10 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33ffff640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd33800eaa0 tx=0x7fd33800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33800cdb0 con 0x7fd340106f10 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd338005020 con 0x7fd340106f10 2024-09-19T05:56:51.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34019d750 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3380116b0 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.130+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3401a5030 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.131+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3401021c0 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.133+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd338005560 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.133+0000 7fd33cff9640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 0x7fd32c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.133+0000 7fd33f7fe640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 0x7fd32c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.134+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fd3380cff20 con 0x7fd340106f10 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.134+0000 7fd33f7fe640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 0x7fd32c0a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd330005ab0 tx=0x7fd330005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:51.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.138+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd338016030 con 0x7fd340106f10 2024-09-19T05:56:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.349+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd340109b60 con 0x7fd340106f10 2024-09-19T05:56:51.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.350+0000 7fd33cff9640 1 -- 172.21.15.38:0/2962689191 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fd33809c820 con 0x7fd340106f10 2024-09-19T05:56:51.353 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:56:51.353 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:56:51.353 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:56:51.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 msgr2=0x7fd32c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:51.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 0x7fd32c0a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd330005ab0 tx=0x7fd330005a20 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 msgr2=0x7fd34019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd33800eaa0 tx=0x7fd33800ef60 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 shutdown_connections 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fd32c0a3240 0x7fd32c0a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd340106f10 0x7fd34019cd80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd340102790 0x7fd3401989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 --2- 172.21.15.38:0/2962689191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd340101de0 0x7fd3401984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.352+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 >> 172.21.15.38:0/2962689191 conn(0x7fd3400fd8c0 msgr2=0x7fd3400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.353+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 shutdown_connections 2024-09-19T05:56:51.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:51.353+0000 7fd346232640 1 -- 172.21.15.38:0/2962689191 wait complete. 2024-09-19T05:56:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:51 smithi038 ceph-mon[31227]: pgmap v821: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:51 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2962689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:51 smithi179 ceph-mon[35781]: pgmap v821: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:51 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2962689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:51 smithi047 ceph-mon[35327]: pgmap v821: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:51.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:51 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2962689191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:52.154 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 563cc35d --release --await' 2024-09-19T05:56:52.154 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id 563cc35d --release --await 2024-09-19T05:56:52.417 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.918+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/3333703553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 msgr2=0x7f6d481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.918+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/3333703553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d481021c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009900 tx=0x7f6d3802a410 comp rx=0 tx=0).stop 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/3333703553 shutdown_connections 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/3333703553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48106f10 0x7f6d4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/3333703553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48102790 0x7f6d481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/3333703553 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d481021c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/3333703553 >> 172.21.15.38:0/3333703553 conn(0x7f6d480fd8c0 msgr2=0x7f6d480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/3333703553 shutdown_connections 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/3333703553 wait complete. 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 Processor -- start 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.919+0000 7f6d4e1c5640 1 -- start start 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48102790 0x7f6d481988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48106f10 0x7f6d4819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48111f70 con 0x7f6d48101de0 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48111df0 con 0x7f6d48106f10 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d4e1c5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481120f0 con 0x7f6d48102790 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58894/0 (socket says 172.21.15.38:58894) 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 -- 172.21.15.38:0/2528891517 learned_addr learned my addr 172.21.15.38:0/2528891517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d47fff640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48106f10 0x7f6d4819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48102790 0x7f6d481988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 -- 172.21.15.38:0/2528891517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48102790 msgr2=0x7f6d481988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48102790 0x7f6d481988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 -- 172.21.15.38:0/2528891517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48106f10 msgr2=0x7f6d4819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48106f10 0x7f6d4819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 -- 172.21.15.38:0/2528891517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4819d340 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.920+0000 7f6d477fe640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6d38006010 tx=0x7f6d380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.921+0000 7f6d44ff9640 1 -- 172.21.15.38:0/2528891517 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3803f030 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.921+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4819d5d0 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.921+0000 7f6d44ff9640 1 -- 172.21.15.38:0/2528891517 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d38003ce0 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.921+0000 7f6d44ff9640 1 -- 172.21.15.38:0/2528891517 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38035070 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.921+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a4f10 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d44ff9640 1 -- 172.21.15.38:0/2528891517 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6d38042030 con 0x7f6d48101de0 2024-09-19T05:56:52.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 Processor -- start 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 -- start start 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9060 0x55cba11f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1203d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cba115e620 con 0x55cba1204240 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cba1105e50 con 0x55cba11f9980 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.923+0000 7f6d5059e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cba115a5e0 con 0x55cba11f9060 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58898/0 (socket says 172.21.15.38:58898) 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 -- 172.21.15.38:0/2364663145 learned_addr learned my addr 172.21.15.38:0/2364663145 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d47fff640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9060 0x55cba11f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d477fe640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1203d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d44ff9640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 0x7f6d300a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 -- 172.21.15.38:0/2364663145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9060 msgr2=0x55cba11f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9060 0x55cba11f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 -- 172.21.15.38:0/2364663145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 msgr2=0x55cba1203d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1203d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 -- 172.21.15.38:0/2364663145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba10f6d40 con 0x55cba1204240 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d477fe640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1203d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d47fff640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 0x7f6d300a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d44ff9640 1 -- 172.21.15.38:0/2528891517 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6d380effd0 con 0x7f6d48101de0 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00aca0 tx=0x7f6d3c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199202aa9b92a250 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d2dffb640 1 -- 172.21.15.38:0/2364663145 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c01d070 con 0x55cba1204240 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d2dffb640 1 -- 172.21.15.38:0/2364663145 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c018b20 con 0x55cba1204240 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.924+0000 7f6d2dffb640 1 -- 172.21.15.38:0/2364663145 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c010cb0 con 0x55cba1204240 2024-09-19T05:56:52.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- 172.21.15.38:0/2364663145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 msgr2=0x55cba1206630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00aca0 tx=0x7f6d3c00f790 comp rx=0 tx=0).stop 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d47fff640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 0x7f6d300a5780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- 172.21.15.38:0/2364663145 shutdown_connections 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba1204240 0x55cba1206630 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1203d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 --2- 172.21.15.38:0/2364663145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9060 0x55cba11f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- 172.21.15.38:0/2364663145 >> 172.21.15.38:0/2364663145 conn(0x55cba11f55f0 msgr2=0x55cba11f59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- 172.21.15.38:0/2364663145 shutdown_connections 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- 172.21.15.38:0/2364663145 wait complete. 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.925+0000 7f6d5059e740 1 -- start start 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 Processor -- start 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba1201680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba1201bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cba1206e10 con 0x55cba11f9060 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cba10e6f10 con 0x55cba1204240 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cba1206f90 con 0x55cba11f9980 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d5059e740 10 client.0 _ll_register_callbacks cb 0x55cba110adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58900/0 (socket says 172.21.15.38:58900) 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 -- 172.21.15.38:0/950585940 learned_addr learned my addr 172.21.15.38:0/950585940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba1201680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d47fff640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba1201bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 -- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 msgr2=0x55cba1201680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba1201680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 -- 172.21.15.38:0/950585940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 msgr2=0x55cba1201bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba1201bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d477fe640 1 -- 172.21.15.38:0/950585940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba0e6d580 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.926+0000 7f6d47fff640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba1201bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d477fe640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6d380b8690 tx=0x7f6d38003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07c7233326bddf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38049050 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6d380f2110 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c018b20 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d380bbd70 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba0f37960 con 0x55cba11f9060 2024-09-19T05:56:52.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c018b20 con 0x55cba11f9060 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6d3803bc40 con 0x55cba11f9060 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 10 client.16050 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 20 client.16050 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba1213d60 0x55cba12161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 4 client.16050 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.927+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cba116bb90 con 0x55cba1213d60 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.928+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba1213d60 0x55cba12161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.928+0000 7f6d26ffd640 10 client.16050 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.928+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba1213d60 0x55cba12161d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927a30a2de37d9e9 server_cookie=f40b9b0c7bb60394 in_seq=0 out_seq=0 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d26ffd640 1 -- 172.21.15.38:0/950585940 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cba116bb90 con 0x55cba1213d60 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d26ffd640 10 client.16050 handle_command_reply: tid=0 2024-09-19T05:56:52.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 client.16050 shutdown 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 msgr2=0x55cba1201140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6d380b8690 tx=0x7f6d38003f50 comp rx=0 tx=0).stop 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 shutdown_connections 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba1213d60 0x55cba12161d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba1201bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba1201680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d26ffd640 0 client.16050 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 --2- 172.21.15.38:0/950585940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1201140 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 >> 172.21.15.38:0/950585940 conn(0x55cba11f55f0 msgr2=0x55cba11fb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 shutdown_connections 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 1 -- 172.21.15.38:0/950585940 wait complete. 2024-09-19T05:56:52.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.929+0000 7f6d5059e740 20 client.16050 trim_cache size 0 max 16384 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.946+0000 7f6d5059e740 1 Processor -- start 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.946+0000 7f6d5059e740 1 -- start start 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.946+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba121bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba121bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d5059e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481aed00 con 0x55cba11f9060 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d5059e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481b0840 con 0x55cba1204240 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d5059e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481af3d0 con 0x55cba11f9980 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58904/0 (socket says 172.21.15.38:58904) 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 -- 172.21.15.38:0/3580543789 learned_addr learned my addr 172.21.15.38:0/3580543789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba121bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 -- 172.21.15.38:0/3580543789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 msgr2=0x55cba121bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba121bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 -- 172.21.15.38:0/3580543789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 msgr2=0x55cba121bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d47fff640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba121bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba121bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 -- 172.21.15.38:0/3580543789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba11f97b0 con 0x55cba11f9060 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d47fff640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba121bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d477fe640 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6d380bc1c0 tx=0x7f6d380ba740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26abd6a31cbc96f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d2d7fa640 1 -- 172.21.15.38:0/3580543789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38049070 con 0x55cba11f9060 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d2d7fa640 1 -- 172.21.15.38:0/3580543789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3803bc40 con 0x55cba11f9060 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.947+0000 7f6d2d7fa640 1 -- 172.21.15.38:0/3580543789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380044e0 con 0x55cba11f9060 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 -- 172.21.15.38:0/3580543789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 msgr2=0x55cba11fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6d380bc1c0 tx=0x7f6d380ba740 comp rx=0 tx=0).stop 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 -- 172.21.15.38:0/3580543789 shutdown_connections 2024-09-19T05:56:52.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba1204240 0x55cba121bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba11f9980 0x55cba121bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 --2- 172.21.15.38:0/3580543789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba11fb1c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 -- 172.21.15.38:0/3580543789 >> 172.21.15.38:0/3580543789 conn(0x55cba11f55f0 msgr2=0x55cba11faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 -- 172.21.15.38:0/3580543789 shutdown_connections 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.948+0000 7f6d5059e740 1 -- 172.21.15.38:0/3580543789 wait complete. 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 -- start start 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 Processor -- start 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1209090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba1204240 0x55cba1276a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481b0170 con 0x55cba11f9060 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481afaa0 con 0x55cba11f9980 2024-09-19T05:56:52.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481acce0 con 0x55cba1204240 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d5059e740 10 client.0 _ll_register_callbacks cb 0x7f6d380b8550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58914/0 (socket says 172.21.15.38:58914) 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d477fe640 1 -- 172.21.15.38:0/4266384803 learned_addr learned my addr 172.21.15.38:0/4266384803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1209090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.949+0000 7f6d47fff640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba1204240 0x55cba1276a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 -- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba1204240 msgr2=0x55cba1276a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba1204240 0x55cba1276a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 -- 172.21.15.38:0/4266384803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 msgr2=0x55cba1209090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1209090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 -- 172.21.15.38:0/4266384803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba1277240 con 0x55cba11f9060 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d477fe640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6d38032c00 tx=0x7f6d380bc9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206646e533d851ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38049070 con 0x55cba11f9060 2024-09-19T05:56:52.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6d380f3380 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3803bc40 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d380bbb90 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d5059e740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba1277240 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.950+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3803bc40 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6d380bc490 con 0x55cba11f9060 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 10 client.16056 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 20 client.16056 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba127f860 0x55cba1281c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 4 client.16056 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "563cc35d", "release": true, "await": true}] 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "563cc35d", "release": true, "await": true}) -- 0x55cba11660a0 con 0x55cba127f860 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba127f860 0x55cba1281c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:52.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d46ffd640 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba127f860 0x55cba1281c50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109686c9d27ca70f server_cookie=3c4b70b4887bb2d2 in_seq=0 out_seq=0 2024-09-19T05:56:52.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.951+0000 7f6d26ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:52.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.955+0000 7f6d26ffd640 1 -- 172.21.15.38:0/4266384803 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cba11660a0 con 0x55cba127f860 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.955+0000 7f6d26ffd640 10 client.16056 handle_command_reply: tid=0 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 client.16056 shutdown 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 msgr2=0x55cba1208b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6d38032c00 tx=0x7f6d380bc9b0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 shutdown_connections 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x55cba127f860 0x55cba1281c50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cba1204240 0x55cba1276a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d26ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cba11f9980 0x55cba1209090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 --2- 172.21.15.38:0/4266384803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cba11f9060 0x55cba1208b50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 >> 172.21.15.38:0/4266384803 conn(0x55cba11f55f0 msgr2=0x55cba11f4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 shutdown_connections 2024-09-19T05:56:52.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 1 -- 172.21.15.38:0/4266384803 wait complete. 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.956+0000 7f6d5059e740 20 client.16056 trim_cache size 0 max 16384 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 52, 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "563cc35d": { 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 52, 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:56:52.971 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.4, 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:52.972 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:56:52.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 msgr2=0x7f6d300a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 0x7f6d300a5780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40004a40 comp rx=0 tx=0).stop 2024-09-19T05:56:52.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 msgr2=0x7f6d48198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:52.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6d38006010 tx=0x7f6d380042f0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 shutdown_connections 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6d300a32c0 0x7f6d300a5780 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d48106f10 0x7f6d4819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d48102790 0x7f6d481988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 --2- 172.21.15.38:0/2528891517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d48101de0 0x7f6d48198390 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 >> 172.21.15.38:0/2528891517 conn(0x7f6d480fd8c0 msgr2=0x7f6d480feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 shutdown_connections 2024-09-19T05:56:52.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:52.957+0000 7f6d4e1c5640 1 -- 172.21.15.38:0/2528891517 wait complete. 2024-09-19T05:56:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:53 smithi038 ceph-mon[31227]: pgmap v822: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:53.691 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '563cc35d', total seconds elapsed: 32.8 2024-09-19T05:56:53.691 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T05:56:53.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:56:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:53 smithi179 ceph-mon[35781]: pgmap v822: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:53.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:53 smithi047 ceph-mon[35327]: pgmap v822: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 -- 172.21.15.38:0/509108286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824101de0 msgr2=0x7fb8241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 --2- 172.21.15.38:0/509108286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824101de0 0x7fb8241021c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb814009900 tx=0x7fb81402a410 comp rx=0 tx=0).stop 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 -- 172.21.15.38:0/509108286 shutdown_connections 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 --2- 172.21.15.38:0/509108286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824106f10 0x7fb82410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 --2- 172.21.15.38:0/509108286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 0x7fb8241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 --2- 172.21.15.38:0/509108286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824101de0 0x7fb8241021c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 -- 172.21.15.38:0/509108286 >> 172.21.15.38:0/509108286 conn(0x7fb8240fd8c0 msgr2=0x7fb8240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 -- 172.21.15.38:0/509108286 shutdown_connections 2024-09-19T05:56:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.473+0000 7fb829e2d640 1 -- 172.21.15.38:0/509108286 wait complete. 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 Processor -- start 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 -- start start 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 0x7fb824198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 0x7fb824198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111f70 con 0x7fb824106f10 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111df0 con 0x7fb824101de0 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb829e2d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241120f0 con 0x7fb824102790 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb823fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58924/0 (socket says 172.21.15.38:58924) 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb823fff640 1 -- 172.21.15.38:0/2021982972 learned_addr learned my addr 172.21.15.38:0/2021982972 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb81bfff640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 0x7fb824198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.474+0000 7fb8237fe640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 0x7fb824198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 -- 172.21.15.38:0/2021982972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 msgr2=0x7fb824198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 0x7fb824198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 -- 172.21.15.38:0/2021982972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 msgr2=0x7fb824198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 0x7fb824198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 -- 172.21.15.38:0/2021982972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82419d400 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb8237fe640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 0x7fb824198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb823fff640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00eaa0 tx=0x7fb81c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00cdb0 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c005020 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82419d6f0 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0116b0 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.475+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a4fd0 con 0x7fb824106f10 2024-09-19T05:56:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.476+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8241021c0 con 0x7fb824106f10 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.477+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb81c005560 con 0x7fb824106f10 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.478+0000 7fb8217fa640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 0x7fb8100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.478+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fb81c0cfee0 con 0x7fb824106f10 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.478+0000 7fb8237fe640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 0x7fb8100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.478+0000 7fb8237fe640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 0x7fb8100a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c00 tx=0x7fb814032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:54.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.482+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb81c016030 con 0x7fb824106f10 2024-09-19T05:56:54.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.698+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb82402d7c0 con 0x7fb824106f10 2024-09-19T05:56:54.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.698+0000 7fb8217fa640 1 -- 172.21.15.38:0/2021982972 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fb81c09c840 con 0x7fb824106f10 2024-09-19T05:56:54.701 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:56:54.701 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:56:54.702 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:56:54.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 msgr2=0x7fb8100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:54.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 0x7fb8100a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c00 tx=0x7fb814032c70 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 msgr2=0x7fb82419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00eaa0 tx=0x7fb81c00ef60 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 shutdown_connections 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.700+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fb8100a3240 0x7fb8100a5700 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb824106f10 0x7fb82419cd20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb824102790 0x7fb824198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 --2- 172.21.15.38:0/2021982972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb824101de0 0x7fb824198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 >> 172.21.15.38:0/2021982972 conn(0x7fb8240fd8c0 msgr2=0x7fb8240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 shutdown_connections 2024-09-19T05:56:54.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:54.701+0000 7fb829e2d640 1 -- 172.21.15.38:0/2021982972 wait complete. 2024-09-19T05:56:55.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T05:56:55.396 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T05:56:55.656 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:56:55.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:55 smithi038 ceph-mon[31227]: pgmap v823: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:55.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:55 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2021982972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:55 smithi179 ceph-mon[35781]: pgmap v823: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:55 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2021982972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:55.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:55 smithi047 ceph-mon[35327]: pgmap v823: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-19T05:56:55.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:55 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2021982972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 -- 172.21.15.38:0/1265830035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe798074620 msgr2=0x7fe798074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1265830035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe798074620 0x7fe798074a00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe79402a480 comp rx=0 tx=0).stop 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 -- 172.21.15.38:0/1265830035 shutdown_connections 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1265830035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79810e3d0 0x7fe798110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1265830035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074f40 0x7fe79810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1265830035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe798074620 0x7fe798074a00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 -- 172.21.15.38:0/1265830035 >> 172.21.15.38:0/1265830035 conn(0x7fe7980fd8b0 msgr2=0x7fe7980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 -- 172.21.15.38:0/1265830035 shutdown_connections 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.226+0000 7fe7a005a640 1 -- 172.21.15.38:0/1265830035 wait complete. 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 Processor -- start 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 -- start start 2024-09-19T05:56:56.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074620 0x7fe79819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 0x7fe79819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe798114160 con 0x7fe79810e3d0 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe798113fe0 con 0x7fe798074f40 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe7a005a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981142e0 con 0x7fe798074620 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79e5f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79e5f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58946/0 (socket says 172.21.15.38:58946) 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79e5f7640 1 -- 172.21.15.38:0/1898959595 learned_addr learned my addr 172.21.15.38:0/1898959595 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074620 0x7fe79819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 0x7fe79819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79e5f7640 1 -- 172.21.15.38:0/1898959595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074620 msgr2=0x7fe79819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.227+0000 7fe79e5f7640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074620 0x7fe79819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe79e5f7640 1 -- 172.21.15.38:0/1898959595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 msgr2=0x7fe79819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe79e5f7640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 0x7fe79819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe79e5f7640 1 -- 172.21.15.38:0/1898959595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79819f630 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 0x7fe79819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe79e5f7640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe790009fd0 tx=0x7fe79000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe78effd640 1 -- 172.21.15.38:0/1898959595 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790017070 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe78effd640 1 -- 172.21.15.38:0/1898959595 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe790003e50 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe78effd640 1 -- 172.21.15.38:0/1898959595 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900106c0 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79819f920 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.228+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a7200 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.230+0000 7fe78effd640 1 -- 172.21.15.38:0/1898959595 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe790003970 con 0x7fe79810e3d0 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.230+0000 7fe7a2433740 1 Processor -- start 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 -- start start 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e1012c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5563e0f77160 con 0x5563e1013170 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5563e0f1e720 con 0x5563e1012850 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe7a2433740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5563e0f73120 con 0x5563e101da30 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58958/0 (socket says 172.21.15.38:58958) 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 -- 172.21.15.38:0/3501568610 learned_addr learned my addr 172.21.15.38:0/3501568610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e1012c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe78effd640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 0x7fe7840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 -- 172.21.15.38:0/3501568610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 msgr2=0x5563e101fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 -- 172.21.15.38:0/3501568610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 msgr2=0x5563e1012c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e1012c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79ddf6640 1 -- 172.21.15.38:0/3501568610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e0ce8ba0 con 0x5563e1013170 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79e5f7640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 0x7fe7840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.231+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e1012c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe78effd640 1 -- 172.21.15.38:0/1898959595 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe7900cf970 con 0x7fe79810e3d0 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe794005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a75da7f14e1db28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe783fff640 1 -- 172.21.15.38:0/3501568610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794040070 con 0x5563e1013170 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe783fff640 1 -- 172.21.15.38:0/3501568610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7940319f0 con 0x5563e1013170 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe79e5f7640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 0x7fe7840a5780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe7900985a0 tx=0x7fe790066ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.232+0000 7fe783fff640 1 -- 172.21.15.38:0/3501568610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794035040 con 0x5563e1013170 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- 172.21.15.38:0/3501568610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 msgr2=0x5563e101d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe794005e80 comp rx=0 tx=0).stop 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- 172.21.15.38:0/3501568610 shutdown_connections 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1013170 0x5563e101d4f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3501568610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e1012c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- 172.21.15.38:0/3501568610 >> 172.21.15.38:0/3501568610 conn(0x5563e100ede0 msgr2=0x5563e100f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- 172.21.15.38:0/3501568610 shutdown_connections 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- 172.21.15.38:0/3501568610 wait complete. 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 -- start start 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.233+0000 7fe7a2433740 1 Processor -- start 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e101ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5563e1020600 con 0x5563e1012850 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5563e0eff900 con 0x5563e1013170 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5563e1020780 con 0x5563e101da30 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 10 client.0 _ll_register_callbacks cb 0x5563e0efd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58960/0 (socket says 172.21.15.38:58960) 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 -- 172.21.15.38:0/3130660494 learned_addr learned my addr 172.21.15.38:0/3130660494 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:56.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e101ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 -- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 msgr2=0x5563e101b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 -- 172.21.15.38:0/3130660494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 msgr2=0x5563e101ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e101ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 -- 172.21.15.38:0/3130660494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563e0f3fc50 con 0x5563e1012850 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e101ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe7880057c0 tx=0x7fe788004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70332eda9fc89ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788017070 con 0x5563e1012850 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe788004020 con 0x5563e1012850 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.234+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7940319f0 con 0x5563e1012850 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe788022050 con 0x5563e1012850 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:56.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563e102c330 con 0x5563e1012850 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7940319f0 con 0x5563e1012850 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe78800f040 con 0x5563e1012850 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 10 client.16074 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 20 client.16074 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e102d740 0x5563e102fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 4 client.16074 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563e0f84890 con 0x5563e102d740 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.235+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e102d740 0x5563e102fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.236+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e102d740 0x5563e102fb30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfe164ec6f693 server_cookie=dafa15a1c55f97ef in_seq=0 out_seq=0 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.236+0000 7fe7817fa640 10 client.16074 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.236+0000 7fe7817fa640 1 -- 172.21.15.38:0/3130660494 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563e0f84890 con 0x5563e102d740 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.236+0000 7fe7817fa640 10 client.16074 handle_command_reply: tid=0 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.236+0000 7fe7a2433740 1 client.16074 shutdown 2024-09-19T05:56:56.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 msgr2=0x5563e101a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe7880057c0 tx=0x7fe788004eb0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 shutdown_connections 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e102d740 0x5563e102fb30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e101da30 0x5563e101b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7817fa640 0 client.16074 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e101ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 --2- 172.21.15.38:0/3130660494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e1012850 0x5563e101a8f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 >> 172.21.15.38:0/3130660494 conn(0x5563e100ede0 msgr2=0x5563e1014b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 shutdown_connections 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 1 -- 172.21.15.38:0/3130660494 wait complete. 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.237+0000 7fe7a2433740 20 client.16074 trim_cache size 0 max 16384 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 Processor -- start 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 -- start start 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e10353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1013170 0x5563e10358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b0f60 con 0x5563e101da30 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b2aa0 con 0x5563e1012850 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe7a2433740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b1630 con 0x5563e1013170 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58970/0 (socket says 172.21.15.38:58970) 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 -- 172.21.15.38:0/2544484706 learned_addr learned my addr 172.21.15.38:0/2544484706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 -- 172.21.15.38:0/2544484706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1013170 msgr2=0x5563e10358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1013170 0x5563e10358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 -- 172.21.15.38:0/2544484706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 msgr2=0x5563e10353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e10353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.254+0000 7fe79d5f5640 1 -- 172.21.15.38:0/2544484706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e100ec10 con 0x5563e101da30 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe788001eb0 tx=0x7fe7880056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb89334becbd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7837fe640 1 -- 172.21.15.38:0/2544484706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788017040 con 0x5563e101da30 2024-09-19T05:56:56.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7837fe640 1 -- 172.21.15.38:0/2544484706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78801f030 con 0x5563e101da30 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7837fe640 1 -- 172.21.15.38:0/2544484706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788026850 con 0x5563e101da30 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 -- 172.21.15.38:0/2544484706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 msgr2=0x5563e1017b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe788001eb0 tx=0x7fe7880056a0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 -- 172.21.15.38:0/2544484706 shutdown_connections 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e1017b70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1013170 0x5563e10358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 --2- 172.21.15.38:0/2544484706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1012850 0x5563e10353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.255+0000 7fe7a2433740 1 -- 172.21.15.38:0/2544484706 >> 172.21.15.38:0/2544484706 conn(0x5563e100ede0 msgr2=0x5563e1014ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- 172.21.15.38:0/2544484706 shutdown_connections 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- 172.21.15.38:0/2544484706 wait complete. 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- start start 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 Processor -- start 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1012850 0x5563e1025bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e10260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b23d0 con 0x5563e101da30 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b1d00 con 0x5563e1013170 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981aef40 con 0x5563e1012850 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe7a2433740 10 client.0 _ll_register_callbacks cb 0x7fe788005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.256+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58986/0 (socket says 172.21.15.38:58986) 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 -- 172.21.15.38:0/1735654569 learned_addr learned my addr 172.21.15.38:0/1735654569 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e10260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1012850 0x5563e1025bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 -- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1012850 msgr2=0x5563e1025bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1012850 0x5563e1025bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 -- 172.21.15.38:0/1735654569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 msgr2=0x5563e10260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e10260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 -- 172.21.15.38:0/1735654569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563e10230e0 con 0x5563e101da30 2024-09-19T05:56:56.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe79d5f5640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe78800bfe0 tx=0x7fe788026430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eec25af478c149a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7817fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788017070 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe788011b20 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78801f030 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe788004020 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7a2433740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563e1098db0 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.257+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78801f030 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe7880264d0 con 0x5563e101da30 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 10 client.16080 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:56:56.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 20 client.16080 populate_metadata read hostname 'smithi038' 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 54, 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "ba019c5b": { 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 54, 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:56.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.4, 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:56:56.345 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e10993d0 0x5563e109b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 4 client.16080 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x5563e0f7ebe0 con 0x5563e10993d0 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e10993d0 0x5563e109b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe7817fa640 10 client.16080 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.258+0000 7fe79ddf6640 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e10993d0 0x5563e109b7c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba87f4d17db0ad server_cookie=ee1a48f4c6f1fa8b in_seq=0 out_seq=0 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7817fa640 1 -- 172.21.15.38:0/1735654569 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5563e0f7ebe0 con 0x5563e10993d0 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7817fa640 10 client.16080 handle_command_reply: tid=0 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 client.16080 shutdown 2024-09-19T05:56:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 msgr2=0x5563e10229a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe78800bfe0 tx=0x7fe788026430 comp rx=0 tx=0).stop 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 shutdown_connections 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5563e10993d0 0x5563e109b7c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5563e101da30 0x5563e10229a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7817fa640 0 client.16080 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e1013170 0x5563e10260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 --2- 172.21.15.38:0/1735654569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563e1012850 0x5563e1025bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 >> 172.21.15.38:0/1735654569 conn(0x5563e100ede0 msgr2=0x5563e1028850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 shutdown_connections 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 1 -- 172.21.15.38:0/1735654569 wait complete. 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a2433740 20 client.16080 trim_cache size 0 max 16384 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 msgr2=0x7fe7840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 0x7fe7840a5780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe7900985a0 tx=0x7fe790066ab0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 msgr2=0x7fe79819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe790009fd0 tx=0x7fe79000b260 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 shutdown_connections 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe7840a32c0 0x7fe7840a5780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe79810e3d0 0x7fe79819ef50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe798074f40 0x7fe79819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 --2- 172.21.15.38:0/1898959595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe798074620 0x7fe79819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 >> 172.21.15.38:0/1898959595 conn(0x7fe7980fd8b0 msgr2=0x7fe7980fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:56:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 shutdown_connections 2024-09-19T05:56:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:56:56.269+0000 7fe7a005a640 1 -- 172.21.15.38:0/1898959595 wait complete. 2024-09-19T05:56:57.068 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ba019c5b', quiesce took 3.4 seconds. Will release after: 26.05 2024-09-19T05:56:57.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:57 smithi038 ceph-mon[31227]: pgmap v824: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:56:57.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:57 smithi179 ceph-mon[35781]: pgmap v824: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:56:57.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:57 smithi047 ceph-mon[35327]: pgmap v824: 129 pgs: 129 active+clean; 7.0 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T05:56:59.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:56:59 smithi038 ceph-mon[31227]: pgmap v825: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-19T05:56:59.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:56:59 smithi179 ceph-mon[35781]: pgmap v825: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-19T05:56:59.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:56:59 smithi047 ceph-mon[35327]: pgmap v825: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-09-19T05:57:01.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:01 smithi038 ceph-mon[31227]: pgmap v826: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:01.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:01 smithi179 ceph-mon[35781]: pgmap v826: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:01.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:01 smithi047 ceph-mon[35327]: pgmap v826: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:03.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:03 smithi038 ceph-mon[31227]: pgmap v827: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:03.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:03 smithi179 ceph-mon[35781]: pgmap v827: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:03.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:03 smithi047 ceph-mon[35327]: pgmap v827: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-19T05:57:05.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:05 smithi038 ceph-mon[31227]: pgmap v828: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:05.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:05 smithi179 ceph-mon[35781]: pgmap v828: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:05.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:05 smithi047 ceph-mon[35327]: pgmap v828: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:07.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:07 smithi038 ceph-mon[31227]: pgmap v829: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:57:07.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:07 smithi179 ceph-mon[35781]: pgmap v829: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:57:07.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:07 smithi047 ceph-mon[35327]: pgmap v829: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-19T05:57:09.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:09 smithi038 ceph-mon[31227]: pgmap v830: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:09.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:09 smithi179 ceph-mon[35781]: pgmap v830: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:09.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:09 smithi047 ceph-mon[35327]: pgmap v830: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-09-19T05:57:11.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:11 smithi038 ceph-mon[31227]: pgmap v831: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:57:11.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:11 smithi179 ceph-mon[35781]: pgmap v831: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:57:11.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:11 smithi047 ceph-mon[35327]: pgmap v831: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T05:57:13.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:13 smithi038 ceph-mon[31227]: pgmap v832: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:13.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:13 smithi179 ceph-mon[35781]: pgmap v832: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:13.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:13 smithi047 ceph-mon[35327]: pgmap v832: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:15.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:15 smithi038 ceph-mon[31227]: pgmap v833: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:15.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:15 smithi179 ceph-mon[35781]: pgmap v833: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:15.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:15 smithi047 ceph-mon[35327]: pgmap v833: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T05:57:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:17 smithi038 ceph-mon[31227]: pgmap v834: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:17.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:17 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:17 smithi179 ceph-mon[35781]: pgmap v834: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:17.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:17 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:17.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:17 smithi047 ceph-mon[35327]: pgmap v834: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:17.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:17 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:57:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:57:18.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:57:18.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:57:18.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:57:18.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:57:19.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:19 smithi179 ceph-mon[35781]: pgmap v835: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:57:19.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:19 smithi047 ceph-mon[35327]: pgmap v835: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:57:20.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:19 smithi038 ceph-mon[31227]: pgmap v835: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-19T05:57:21.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:21 smithi179 ceph-mon[35781]: pgmap v836: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:21.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:21 smithi047 ceph-mon[35327]: pgmap v836: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:22.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:21 smithi038 ceph-mon[31227]: pgmap v836: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:23.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:57:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:57:23.833 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:23 smithi038 ceph-mon[31227]: pgmap v837: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:57:23.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:23 smithi179 ceph-mon[35781]: pgmap v837: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 -- 172.21.15.38:0/3567629505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518101de0 msgr2=0x7f75181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 --2- 172.21.15.38:0/3567629505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518101de0 0x7f75181021c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a410 comp rx=0 tx=0).stop 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 -- 172.21.15.38:0/3567629505 shutdown_connections 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 --2- 172.21.15.38:0/3567629505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 0x7f751810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 --2- 172.21.15.38:0/3567629505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518102790 0x7f75181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 --2- 172.21.15.38:0/3567629505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518101de0 0x7f75181021c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 -- 172.21.15.38:0/3567629505 >> 172.21.15.38:0/3567629505 conn(0x7f75180fd8c0 msgr2=0x7f75180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 -- 172.21.15.38:0/3567629505 shutdown_connections 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 -- 172.21.15.38:0/3567629505 wait complete. 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.915+0000 7f751e161640 1 Processor -- start 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 -- start start 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518101de0 0x7f75181984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 0x7f751819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111f70 con 0x7f7518102790 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111df0 con 0x7f7518106f10 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f751e161640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f75181120f0 con 0x7f7518101de0 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:23.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60894/0 (socket says 172.21.15.38:60894) 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 -- 172.21.15.38:0/663131476 learned_addr learned my addr 172.21.15.38:0/663131476 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f75177fe640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518101de0 0x7f75181984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7517fff640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 0x7f751819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 -- 172.21.15.38:0/663131476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518101de0 msgr2=0x7f75181984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518101de0 0x7f75181984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 -- 172.21.15.38:0/663131476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 msgr2=0x7f751819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 0x7f751819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 -- 172.21.15.38:0/663131476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751819d480 con 0x7f7518102790 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.916+0000 7f7516ffd640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d950 tx=0x7f750c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.917+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c017070 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.917+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c009c30 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.917+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c00f6b0 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.917+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751819d770 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.917+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a5050 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.918+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75181021c0 con 0x7f7518102790 2024-09-19T05:57:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.919+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f750c0109d0 con 0x7f7518102790 2024-09-19T05:57:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.919+0000 7f7514ff9640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 0x7f75000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.920+0000 7f75177fe640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 0x7f75000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.920+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f750c0cf920 con 0x7f7518102790 2024-09-19T05:57:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.920+0000 7f75177fe640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 0x7f75000a5780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7508009230 tx=0x7f75080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:23.924+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f750c009de0 con 0x7f7518102790 2024-09-19T05:57:23.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:23 smithi047 ceph-mon[35327]: pgmap v837: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9 op/s 2024-09-19T05:57:24.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.137+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7518109b60 con 0x7f7518102790 2024-09-19T05:57:24.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.138+0000 7f7514ff9640 1 -- 172.21.15.38:0/663131476 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f750c09c220 con 0x7f7518102790 2024-09-19T05:57:24.140 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:57:24.140 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:57:24.140 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 msgr2=0x7f75000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 0x7f75000a5780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7508009230 tx=0x7f75080058d0 comp rx=0 tx=0).stop 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 msgr2=0x7f7518198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d950 tx=0x7f750c00de10 comp rx=0 tx=0).stop 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 shutdown_connections 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f75000a32c0 0x7f75000a5780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7518106f10 0x7f751819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7518102790 0x7f7518198a10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:24.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 --2- 172.21.15.38:0/663131476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7518101de0 0x7f75181984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:24.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 >> 172.21.15.38:0/663131476 conn(0x7f75180fd8c0 msgr2=0x7f75180fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:24.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.140+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 shutdown_connections 2024-09-19T05:57:24.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:24.141+0000 7f751e161640 1 -- 172.21.15.38:0/663131476 wait complete. 2024-09-19T05:57:24.813 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:24 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/663131476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:57:24.845 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id ba019c5b --release --await' 2024-09-19T05:57:24.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --set-id ba019c5b --release --await 2024-09-19T05:57:24.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:24 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/663131476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:57:25.107 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:57:25.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:24 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/663131476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 -- 172.21.15.38:0/2566896298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04100a80 msgr2=0x7f6e04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2566896298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04100a80 0x7f6e04100ee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009930 tx=0x7f6df802a420 comp rx=0 tx=0).stop 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 -- 172.21.15.38:0/2566896298 shutdown_connections 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2566896298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04101420 0x7f6e0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2566896298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04100a80 0x7f6e04100ee0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2566896298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 -- 172.21.15.38:0/2566896298 >> 172.21.15.38:0/2566896298 conn(0x7f6e040fc7a0 msgr2=0x7f6e040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 -- 172.21.15.38:0/2566896298 shutdown_connections 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.596+0000 7f6e09c30640 1 -- 172.21.15.38:0/2566896298 wait complete. 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 Processor -- start 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 -- start start 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04100a80 0x7f6e04198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e0419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04110e80 con 0x7f6e04101420 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04110d00 con 0x7f6e04106a80 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e09c30640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04111000 con 0x7f6e04100a80 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60920/0 (socket says 172.21.15.38:60920) 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.597+0000 7f6e02ffd640 1 -- 172.21.15.38:0/2104253700 learned_addr learned my addr 172.21.15.38:0/2104253700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e03fff640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e0419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e037fe640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04100a80 0x7f6e04198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 -- 172.21.15.38:0/2104253700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04100a80 msgr2=0x7f6e04198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04100a80 0x7f6e04198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 -- 172.21.15.38:0/2104253700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 msgr2=0x7f6e0419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e0419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 -- 172.21.15.38:0/2104253700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0419d3f0 con 0x7f6e04101420 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e03fff640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e0419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009230 tx=0x7f6df8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e00ff9640 1 -- 172.21.15.38:0/2104253700 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df803f030 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0419d680 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e00ff9640 1 -- 172.21.15.38:0/2104253700 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df8003ad0 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e00ff9640 1 -- 172.21.15.38:0/2104253700 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8035070 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.598+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e041a4fc0 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e00ff9640 1 -- 172.21.15.38:0/2104253700 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6df8042030 con 0x7f6e04101420 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 Processor -- start 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 -- start start 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b13630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a6b620 con 0x5654d7b06060 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a12e50 con 0x5654d7b06980 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e0c009740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5654d7a675e0 con 0x5654d7b11240 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60928/0 (socket says 172.21.15.38:60928) 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.601+0000 7f6e03fff640 1 -- 172.21.15.38:0/3549845362 learned_addr learned my addr 172.21.15.38:0/3549845362 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e037fe640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 -- 172.21.15.38:0/3549845362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 msgr2=0x5654d7b13630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b13630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b13630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 -- 172.21.15.38:0/3549845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 msgr2=0x5654d7b10d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b10d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 -- 172.21.15.38:0/3549845362 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7a03d40 con 0x5654d7b06060 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e037fe640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b10d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e00ff9640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 0x7f6dec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e03fff640 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007f40 tx=0x7f6dfc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4b1e5dbc52924 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e037fe640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 0x7f6dec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6e00ff9640 1 -- 172.21.15.38:0/2104253700 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6df80f0270 con 0x7f6e04101420 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6de9ffb640 1 -- 172.21.15.38:0/3549845362 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc014070 con 0x5654d7b06060 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6de9ffb640 1 -- 172.21.15.38:0/3549845362 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dfc004d20 con 0x5654d7b06060 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.602+0000 7f6de9ffb640 1 -- 172.21.15.38:0/3549845362 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc005590 con 0x5654d7b06060 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- 172.21.15.38:0/3549845362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 msgr2=0x5654d7b06440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e037fe640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 0x7f6dec0a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005fd0 tx=0x7f6df4005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007f40 tx=0x7f6dfc00e7e0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- 172.21.15.38:0/3549845362 shutdown_connections 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b10d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 --2- 172.21.15.38:0/3549845362 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b06440 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- 172.21.15.38:0/3549845362 >> 172.21.15.38:0/3549845362 conn(0x5654d7b025f0 msgr2=0x5654d7b029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- 172.21.15.38:0/3549845362 shutdown_connections 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- 172.21.15.38:0/3549845362 wait complete. 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.603+0000 7f6e0c009740 1 -- start start 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 Processor -- start 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b0ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5654d7b18220 con 0x5654d7b06060 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5654d79f3f10 con 0x5654d7b06980 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5654d7b183a0 con 0x5654d7b11240 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e0c009740 10 client.0 _ll_register_callbacks cb 0x5654d7a17db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60930/0 (socket says 172.21.15.38:60930) 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 -- 172.21.15.38:0/970043677 learned_addr learned my addr 172.21.15.38:0/970043677 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e03fff640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b0ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 -- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 msgr2=0x5654d7b0ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b0ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 -- 172.21.15.38:0/970043677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 msgr2=0x5654d7b0e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b0e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.604+0000 7f6e037fe640 1 -- 172.21.15.38:0/970043677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d777a580 con 0x5654d7b06060 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e037fe640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005f80 tx=0x7f6df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1611b9fc734c50c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df401f8f0 con 0x5654d7b06060 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6df401fa90 con 0x5654d7b06060 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dfc004d20 con 0x5654d7b06060 2024-09-19T05:57:25.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6df401fc90 con 0x5654d7b06060 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654d7844960 con 0x5654d7b06060 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dfc004d20 con 0x5654d7b06060 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6df401d4d0 con 0x5654d7b06060 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:57:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 10 client.16098 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 20 client.16098 populate_metadata read hostname 'smithi038' 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b252a0 0x5654d7b276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 4 client.16098 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.605+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d7a78b90 con 0x5654d7b252a0 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.606+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b252a0 0x5654d7b276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.606+0000 7f6de2ffd640 10 client.16098 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.606+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b252a0 0x5654d7b276f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c7610a944e274a server_cookie=9bef35bd7ba7ecc5 in_seq=0 out_seq=0 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6de2ffd640 1 -- 172.21.15.38:0/970043677 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d7a78b90 con 0x5654d7b252a0 2024-09-19T05:57:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6de2ffd640 10 client.16098 handle_command_reply: tid=0 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 client.16098 shutdown 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 msgr2=0x5654d7b0e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005f80 tx=0x7f6df40046c0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 shutdown_connections 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b252a0 0x5654d7b276f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06980 0x5654d7b0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 --2- 172.21.15.38:0/970043677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06060 0x5654d7b0e1f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6de2ffd640 0 client.16098 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 >> 172.21.15.38:0/970043677 conn(0x5654d7b025f0 msgr2=0x5654d7b08320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 shutdown_connections 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 1 -- 172.21.15.38:0/970043677 wait complete. 2024-09-19T05:57:25.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.607+0000 7f6e0c009740 20 client.16098 trim_cache size 0 max 16384 2024-09-19T05:57:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.624+0000 7f6e0c009740 1 Processor -- start 2024-09-19T05:57:25.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 -- start start 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b08b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b2cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041acac0 con 0x5654d7b06980 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041ae600 con 0x5654d7b06060 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e0c009740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041ad190 con 0x5654d7b11240 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60944/0 (socket says 172.21.15.38:60944) 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 -- 172.21.15.38:0/155375597 learned_addr learned my addr 172.21.15.38:0/155375597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e037fe640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b08b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e03fff640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b2cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 -- 172.21.15.38:0/155375597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 msgr2=0x5654d7b2cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b2cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 -- 172.21.15.38:0/155375597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 msgr2=0x5654d7b08b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b08b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 -- 172.21.15.38:0/155375597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d7b067b0 con 0x5654d7b06980 2024-09-19T05:57:25.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e037fe640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b08b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:57:25.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.625+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ed9b0 tx=0x7f6df80b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117f9d7edc7bd750 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6de97fa640 1 -- 172.21.15.38:0/155375597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8049040 con 0x5654d7b06980 2024-09-19T05:57:25.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6de97fa640 1 -- 172.21.15.38:0/155375597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df80bc4d0 con 0x5654d7b06980 2024-09-19T05:57:25.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6de97fa640 1 -- 172.21.15.38:0/155375597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80b8b40 con 0x5654d7b06980 2024-09-19T05:57:25.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 -- 172.21.15.38:0/155375597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 msgr2=0x5654d7b01640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ed9b0 tx=0x7f6df80b8040 comp rx=0 tx=0).stop 2024-09-19T05:57:25.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 -- 172.21.15.38:0/155375597 shutdown_connections 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b11240 0x5654d7b2cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b06980 0x5654d7b01640 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 --2- 172.21.15.38:0/155375597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b08b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 -- 172.21.15.38:0/155375597 >> 172.21.15.38:0/155375597 conn(0x5654d7b025f0 msgr2=0x5654d7b00f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 -- 172.21.15.38:0/155375597 shutdown_connections 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.626+0000 7f6e0c009740 1 -- 172.21.15.38:0/155375597 wait complete. 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 -- start start 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 Processor -- start 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b06980 0x5654d7b19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041adf30 con 0x5654d7b11240 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041ad860 con 0x5654d7b06060 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041aaaa0 con 0x5654d7b06980 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e0c009740 10 client.0 _ll_register_callbacks cb 0x7f6df400be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.627+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60950/0 (socket says 172.21.15.38:60950) 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 -- 172.21.15.38:0/1946823026 learned_addr learned my addr 172.21.15.38:0/1946823026 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e037fe640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e02ffd640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b06980 0x5654d7b19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 -- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b06980 msgr2=0x5654d7b19ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b06980 0x5654d7b19ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 -- 172.21.15.38:0/1946823026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 msgr2=0x5654d7b19590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b19590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 -- 172.21.15.38:0/1946823026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d7b8f130 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007f10 tx=0x7f6dfc005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c133b10f716b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc014040 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f6dfc010a50 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.628+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df80bc4d0 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc010c50 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654d7b90420 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df80bc4d0 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6dfc023070 con 0x5654d7b11240 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:57:25.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 10 client.16104 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 20 client.16104 populate_metadata read hostname 'smithi038' 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b90a30 0x5654d7b92e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 4 client.16104 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ba019c5b", "release": true, "await": true}] 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ba019c5b", "release": true, "await": true}) -- 0x5654d7a730a0 con 0x5654d7b90a30 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.629+0000 7f6e037fe640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b90a30 0x5654d7b92e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.630+0000 7f6e037fe640 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b90a30 0x5654d7b92e50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316cd46a7a289c48 server_cookie=68ac6777faca044d in_seq=0 out_seq=0 2024-09-19T05:57:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.630+0000 7f6de2ffd640 10 client.16104 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6de2ffd640 1 -- 172.21.15.38:0/1946823026 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5654d7a730a0 con 0x5654d7b90a30 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6de2ffd640 10 client.16104 handle_command_reply: tid=0 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 client.16104 shutdown 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 msgr2=0x5654d7b1a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007f10 tx=0x7f6dfc005c70 comp rx=0 tx=0).stop 2024-09-19T05:57:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 shutdown_connections 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5654d7b90a30 0x5654d7b92e50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5654d7b11240 0x5654d7b1a010 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654d7b06980 0x5654d7b19ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6de2ffd640 0 client.16104 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 --2- 172.21.15.38:0/1946823026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5654d7b06060 0x5654d7b19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 >> 172.21.15.38:0/1946823026 conn(0x5654d7b025f0 msgr2=0x5654d7b0e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 shutdown_connections 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 1 -- 172.21.15.38:0/1946823026 wait complete. 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.633+0000 7f6e0c009740 20 client.16104 trim_cache size 0 max 16384 2024-09-19T05:57:25.641 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 56, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "ba019c5b": { 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 56, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-19T05:57:25.642 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 149.4, 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:57:25.643 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 msgr2=0x7f6dec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 0x7f6dec0a5700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005fd0 tx=0x7f6df4005d10 comp rx=0 tx=0).stop 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 msgr2=0x7f6e04198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009230 tx=0x7f6df8004270 comp rx=0 tx=0).stop 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 shutdown_connections 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f6dec0a3240 0x7f6dec0a5700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e04106a80 0x7f6e0419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e04101420 0x7f6e04198980 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 --2- 172.21.15.38:0/2104253700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e04100a80 0x7f6e04198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:57:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 >> 172.21.15.38:0/2104253700 conn(0x7f6e040fc7a0 msgr2=0x7f6e041048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:57:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 shutdown_connections 2024-09-19T05:57:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:57:25.634+0000 7f6e09c30640 1 -- 172.21.15.38:0/2104253700 wait complete. 2024-09-19T05:57:25.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:25 smithi038 ceph-mon[31227]: pgmap v838: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:57:25.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:25 smithi047 ceph-mon[35327]: pgmap v838: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:57:26.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:25 smithi179 ceph-mon[35781]: pgmap v838: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:57:26.417 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ba019c5b', total seconds elapsed: 32.7 2024-09-19T05:57:27.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:27 smithi047 ceph-mon[35327]: pgmap v839: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:28.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:27 smithi038 ceph-mon[31227]: pgmap v839: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:28.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:27 smithi179 ceph-mon[35781]: pgmap v839: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:57:29.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:29 smithi047 ceph-mon[35327]: pgmap v840: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:57:29.999 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:29 smithi179 ceph-mon[35781]: pgmap v840: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:57:30.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:29 smithi038 ceph-mon[31227]: pgmap v840: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:499/600: open fd = 3 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:500/600: open fd = 3 2024-09-19T05:57:31.684 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:501/600: open fd = 3 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:502/600: open fd = 3 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:503/600: open fd = 3 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.685 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:504/600: open fd = 3 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:505/600: open fd = 3 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:506/600: open fd = 3 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.686 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:507/600: open fd = 3 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:508/600: open fd = 3 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:509/600: open fd = 3 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.687 INFO:tasks.workunit.client.0.smithi038.stdout:510/600: open fd = 3 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:511/600: open fd = 3 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:512/600: open fd = 3 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:513/600: open fd = 3 2024-09-19T05:57:31.688 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:514/600: open fd = 3 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:515/600: open fd = 3 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:516/600: open fd = 3 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.689 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:517/600: open fd = 3 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:518/600: open fd = 3 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:519/600: open fd = 3 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.690 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:520/600: open fd = 3 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:521/600: open fd = 3 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:522/600: open fd = 3 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.691 INFO:tasks.workunit.client.0.smithi038.stdout:523/600: open fd = 3 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:524/600: open fd = 3 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:525/600: open fd = 3 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:526/600: open fd = 3 2024-09-19T05:57:31.692 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:527/600: open fd = 3 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:528/600: open fd = 3 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:529/600: open fd = 3 2024-09-19T05:57:31.693 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:530/600: open fd = 3 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:531/600: open fd = 3 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:532/600: open fd = 3 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.694 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:533/600: open fd = 3 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:534/600: open fd = 3 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:535/600: open fd = 3 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.695 INFO:tasks.workunit.client.0.smithi038.stdout:536/600: open fd = 3 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:537/600: open fd = 3 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:538/600: open fd = 3 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.696 INFO:tasks.workunit.client.0.smithi038.stdout:539/600: open fd = 3 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:540/600: open fd = 3 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:541/600: open fd = 3 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.697 INFO:tasks.workunit.client.0.smithi038.stdout:542/600: open fd = 3 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:543/600: open fd = 3 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:544/600: open fd = 3 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:545/600: open fd = 3 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.698 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:546/600: open fd = 3 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:547/600: open fd = 3 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:548/600: open fd = 3 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.699 INFO:tasks.workunit.client.0.smithi038.stdout:549/600: open fd = 3 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:550/600: open fd = 3 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:551/600: open fd = 3 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:552/600: open fd = 3 2024-09-19T05:57:31.700 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:553/600: open fd = 3 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:554/600: open fd = 3 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:555/600: open fd = 3 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.701 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:556/600: open fd = 3 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:557/600: open fd = 3 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:558/600: open fd = 3 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.702 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:559/600: open fd = 3 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:560/600: open fd = 3 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:57:31.703 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:57:32.143 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:31 smithi038 ceph-mon[31227]: pgmap v841: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:32.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:31 smithi179 ceph-mon[35781]: pgmap v841: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:32.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:31 smithi047 ceph-mon[35327]: pgmap v841: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:33.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:32 smithi038 ceph-mon[31227]: pgmap v842: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:33.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:32 smithi179 ceph-mon[35781]: pgmap v842: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:33.225 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:32 smithi047 ceph-mon[35327]: pgmap v842: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-19T05:57:35.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:34 smithi038 ceph-mon[31227]: pgmap v843: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-19T05:57:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:34 smithi179 ceph-mon[35781]: pgmap v843: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-19T05:57:35.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:34 smithi047 ceph-mon[35327]: pgmap v843: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-09-19T05:57:37.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:37 smithi038 ceph-mon[31227]: pgmap v844: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:57:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:37 smithi179 ceph-mon[35781]: pgmap v844: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:57:37.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:37 smithi047 ceph-mon[35327]: pgmap v844: 129 pgs: 129 active+clean; 7.1 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-09-19T05:57:39.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:39 smithi038 ceph-mon[31227]: pgmap v845: 129 pgs: 129 active+clean; 7.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-19T05:57:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:39 smithi179 ceph-mon[35781]: pgmap v845: 129 pgs: 129 active+clean; 7.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-19T05:57:39.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:39 smithi047 ceph-mon[35327]: pgmap v845: 129 pgs: 129 active+clean; 7.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-09-19T05:57:41.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:41 smithi038 ceph-mon[31227]: pgmap v846: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:41.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:41 smithi179 ceph-mon[35781]: pgmap v846: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:41.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:41 smithi047 ceph-mon[35327]: pgmap v846: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:43.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:43 smithi038 ceph-mon[31227]: pgmap v847: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:43 smithi179 ceph-mon[35781]: pgmap v847: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:43.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:43 smithi047 ceph-mon[35327]: pgmap v847: 129 pgs: 129 active+clean; 7.2 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-09-19T05:57:45.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:45 smithi038 ceph-mon[31227]: pgmap v848: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:45 smithi179 ceph-mon[35781]: pgmap v848: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:45.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:45 smithi047 ceph-mon[35327]: pgmap v848: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:47.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:47 smithi038 ceph-mon[31227]: pgmap v849: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:57:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:47 smithi179 ceph-mon[35781]: pgmap v849: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:57:47.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:47 smithi047 ceph-mon[35327]: pgmap v849: 129 pgs: 129 active+clean; 7.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T05:57:49.394 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:49 smithi038 ceph-mon[31227]: pgmap v850: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:57:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:49 smithi179 ceph-mon[35781]: pgmap v850: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:57:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:49 smithi047 ceph-mon[35327]: pgmap v850: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-09-19T05:57:51.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:51 smithi038 ceph-mon[31227]: pgmap v851: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:57:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:51 smithi179 ceph-mon[35781]: pgmap v851: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:57:51.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:51 smithi047 ceph-mon[35327]: pgmap v851: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:57:53.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:53 smithi038 ceph-mon[31227]: pgmap v852: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:57:53.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:53 smithi179 ceph-mon[35781]: pgmap v852: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:57:53.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:53 smithi047 ceph-mon[35327]: pgmap v852: 129 pgs: 129 active+clean; 7.3 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:57:55.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:55 smithi038 ceph-mon[31227]: pgmap v853: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:55.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:55 smithi179 ceph-mon[35781]: pgmap v853: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:55.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:55 smithi047 ceph-mon[35327]: pgmap v853: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-19T05:57:57.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:57 smithi038 ceph-mon[31227]: pgmap v854: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:57:57.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:57 smithi179 ceph-mon[35781]: pgmap v854: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:57:57.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:57 smithi047 ceph-mon[35327]: pgmap v854: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:57:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:57:59 smithi038 ceph-mon[31227]: pgmap v855: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:57:59.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:57:59 smithi179 ceph-mon[35781]: pgmap v855: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:57:59.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:57:59 smithi047 ceph-mon[35327]: pgmap v855: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:01.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:01 smithi038 ceph-mon[31227]: pgmap v856: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:01.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:01 smithi179 ceph-mon[35781]: pgmap v856: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:01.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:01 smithi047 ceph-mon[35327]: pgmap v856: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:03.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:03 smithi038 ceph-mon[31227]: pgmap v857: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:03.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:03 smithi179 ceph-mon[35781]: pgmap v857: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:03.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:03 smithi047 ceph-mon[35327]: pgmap v857: 129 pgs: 129 active+clean; 7.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:05.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:05 smithi038 ceph-mon[31227]: pgmap v858: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:05.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:05 smithi179 ceph-mon[35781]: pgmap v858: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:05.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:05 smithi047 ceph-mon[35327]: pgmap v858: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:07.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:07 smithi038 ceph-mon[31227]: pgmap v859: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:07.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:07 smithi179 ceph-mon[35781]: pgmap v859: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:07.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:07 smithi047 ceph-mon[35327]: pgmap v859: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:09.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:09 smithi038 ceph-mon[31227]: pgmap v860: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:09.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:09 smithi179 ceph-mon[35781]: pgmap v860: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:09.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:09 smithi047 ceph-mon[35327]: pgmap v860: 129 pgs: 129 active+clean; 7.5 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-09-19T05:58:11.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:11 smithi038 ceph-mon[31227]: pgmap v861: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:11.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:11 smithi179 ceph-mon[35781]: pgmap v861: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:11.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:11 smithi047 ceph-mon[35327]: pgmap v861: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-19T05:58:11.980 INFO:tasks.workunit.client.0.smithi038.stdout:561/600: open fd = 3 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:562/600: open fd = 3 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:563/600: open fd = 3 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.981 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:564/600: open fd = 3 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:565/600: open fd = 3 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:566/600: open fd = 3 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.982 INFO:tasks.workunit.client.0.smithi038.stdout:567/600: open fd = 3 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:568/600: open fd = 3 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:569/600: open fd = 3 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.983 INFO:tasks.workunit.client.0.smithi038.stdout:570/600: open fd = 3 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:571/600: open fd = 3 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:572/600: open fd = 3 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:573/600: open fd = 3 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.984 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:574/600: open fd = 3 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:575/600: open fd = 3 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:576/600: open fd = 3 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.985 INFO:tasks.workunit.client.0.smithi038.stdout:577/600: open fd = 3 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:578/600: open fd = 3 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:579/600: open fd = 3 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:580/600: open fd = 3 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.986 INFO:tasks.workunit.client.0.smithi038.stdout:581/600: open fd = 3 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:582/600: open fd = 3 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:583/600: open fd = 3 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:584/600: open fd = 3 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.987 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:585/600: open fd = 3 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:586/600: open fd = 3 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:587/600: open fd = 3 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.988 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:588/600: open fd = 3 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:589/600: open fd = 3 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:590/600: open fd = 3 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.989 INFO:tasks.workunit.client.0.smithi038.stdout:591/600: open fd = 3 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:592/600: open fd = 3 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:593/600: open fd = 3 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:594/600: open fd = 3 2024-09-19T05:58:11.990 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:595/600: open fd = 3 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:596/600: open fd = 3 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:597/600: open fd = 3 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.991 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:598/600: open fd = 3 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:599/600: open fd = 3 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:600/600: open fd = 3 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:write ret = 32 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:pread ret = 64 2024-09-19T05:58:11.992 INFO:tasks.workunit.client.0.smithi038.stdout:OK 2024-09-19T05:58:11.994 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo OK 2024-09-19T05:58:11.994 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:58:11.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:58:12.040 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-19T05:58:12.040 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T05:58:12.357 DEBUG:teuthology.parallel:result is None 2024-09-19T05:58:12.358 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:58:12.391 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:58:12.392 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T05:58:12.405 INFO:tasks.quiescer:joining Quiescers 2024-09-19T05:58:12.405 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T05:58:12.405 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:58:12.687 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 -- 172.21.15.38:0/3307812565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 msgr2=0x7faa1410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 --2- 172.21.15.38:0/3307812565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1410da40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009ca0 tx=0x7faa0c02a950 comp rx=0 tx=0).stop 2024-09-19T05:58:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 -- 172.21.15.38:0/3307812565 shutdown_connections 2024-09-19T05:58:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 --2- 172.21.15.38:0/3307812565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1410da40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 --2- 172.21.15.38:0/3307812565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14102570 0x7faa141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 --2- 172.21.15.38:0/3307812565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14100e20 0x7faa14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 -- 172.21.15.38:0/3307812565 >> 172.21.15.38:0/3307812565 conn(0x7faa140fc7d0 msgr2=0x7faa140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 -- 172.21.15.38:0/3307812565 shutdown_connections 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.249+0000 7faa18fd2640 1 -- 172.21.15.38:0/3307812565 wait complete. 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 Processor -- start 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 -- start start 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14100e20 0x7faa14198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14102570 0x7faa14198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faa14110f80 con 0x7faa14102ef0 2024-09-19T05:58:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa14110e00 con 0x7faa14102570 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa18fd2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa14111100 con 0x7faa14100e20 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa12d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa12d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46658/0 (socket says 172.21.15.38:46658) 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.250+0000 7faa12d9d640 1 -- 172.21.15.38:0/1820892485 learned_addr learned my addr 172.21.15.38:0/1820892485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa1259c640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14100e20 0x7faa14198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa11d9b640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14102570 0x7faa14198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 -- 172.21.15.38:0/1820892485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14100e20 msgr2=0x7faa14198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14100e20 0x7faa14198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 -- 172.21.15.38:0/1820892485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14102570 msgr2=0x7faa14198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14102570 0x7faa14198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 -- 172.21.15.38:0/1820892485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1419d400 con 0x7faa14102ef0 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa12d9d640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02ae70 tx=0x7faa0c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c034040 con 0x7faa14102ef0 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1419d690 con 0x7faa14102ef0 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0c030920 con 0x7faa14102ef0 2024-09-19T05:58:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c045480 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.251+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141a4fd0 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.252+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa14101200 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.255+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faa0c037020 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.255+0000 7faa037fe640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 0x7fa9fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.256+0000 7faa1259c640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 0x7fa9fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.256+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7faa0c0f0ce0 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.256+0000 7faa1259c640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 0x7fa9fc0a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa0400bbe0 tx=0x7faa0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:13.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.258+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faa0c081210 con 0x7faa14102ef0 2024-09-19T05:58:13.417 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:13 smithi038 ceph-mon[31227]: pgmap v862: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:13.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.471+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa1410a0a0 con 0x7faa14102ef0 2024-09-19T05:58:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.472+0000 7faa037fe640 1 -- 172.21.15.38:0/1820892485 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7faa0c0bd5e0 con 0x7faa14102ef0 2024-09-19T05:58:13.474 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:13.474 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:58:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:58:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 msgr2=0x7fa9fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 0x7fa9fc0a5700 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa0400bbe0 tx=0x7faa0400b390 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 msgr2=0x7faa1419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa0c02ae70 tx=0x7faa0c02f5d0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 shutdown_connections 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fa9fc0a3240 0x7fa9fc0a5700 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa14102ef0 0x7faa1419cd20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.474+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa14102570 0x7faa14198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.475+0000 7faa18fd2640 1 --2- 172.21.15.38:0/1820892485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa14100e20 0x7faa14198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.475+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 >> 172.21.15.38:0/1820892485 conn(0x7faa140fc7d0 msgr2=0x7faa1406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.475+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 shutdown_connections 2024-09-19T05:58:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:13.475+0000 7faa18fd2640 1 -- 172.21.15.38:0/1820892485 wait complete. 2024-09-19T05:58:13.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:13 smithi179 ceph-mon[35781]: pgmap v862: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:13.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:13 smithi047 ceph-mon[35327]: pgmap v862: 129 pgs: 129 active+clean; 7.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-19T05:58:14.250 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --cancel --all' 2024-09-19T05:58:14.250 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24496 quiesce db --cancel --all 2024-09-19T05:58:14.512 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:14.633 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:14 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1820892485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:14.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:14 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1820892485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:14.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:14 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1820892485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:15.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 -- 172.21.15.38:0/1821604017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924102790 msgr2=0x7fe9241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 --2- 172.21.15.38:0/1821604017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924102790 0x7fe9241067e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe91802a410 comp rx=0 tx=0).stop 2024-09-19T05:58:15.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 -- 172.21.15.38:0/1821604017 shutdown_connections 2024-09-19T05:58:15.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 --2- 172.21.15.38:0/1821604017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924106f10 0x7fe92410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 --2- 172.21.15.38:0/1821604017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924102790 0x7fe9241067e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 --2- 172.21.15.38:0/1821604017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924101de0 0x7fe9241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 -- 172.21.15.38:0/1821604017 >> 172.21.15.38:0/1821604017 conn(0x7fe9240fd8c0 msgr2=0x7fe9240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 -- 172.21.15.38:0/1821604017 shutdown_connections 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.055+0000 7fe929a29640 1 -- 172.21.15.38:0/1821604017 wait complete. 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 Processor -- start 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 -- start start 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924101de0 0x7fe924198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924102790 0x7fe924198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe924111f30 con 0x7fe924106f10 2024-09-19T05:58:15.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe924111db0 con 0x7fe924102790 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe929a29640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241120b0 con 0x7fe924101de0 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe9227fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924102790 0x7fe924198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46680/0 (socket says 172.21.15.38:46680) 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe9237fe640 1 -- 172.21.15.38:0/4102200184 learned_addr learned my addr 172.21.15.38:0/4102200184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.056+0000 7fe922ffd640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924101de0 0x7fe924198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 -- 172.21.15.38:0/4102200184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924101de0 msgr2=0x7fe924198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924101de0 0x7fe924198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 -- 172.21.15.38:0/4102200184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924102790 msgr2=0x7fe924198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924102790 0x7fe924198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 -- 172.21.15.38:0/4102200184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92419d3c0 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe9237fe640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00cce0 tx=0x7fe91c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe928a27640 1 -- 172.21.15.38:0/4102200184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f450 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe928a27640 1 -- 172.21.15.38:0/4102200184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c00fad0 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92419d6b0 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe928a27640 1 -- 172.21.15.38:0/4102200184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c018c10 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.057+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9241a4f90 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.059+0000 7fe928a27640 1 -- 172.21.15.38:0/4102200184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe91c00f5f0 con 0x7fe924106f10 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 Processor -- start 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 -- start start 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1858a00 0x5625f1858de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe928a27640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 0x7fe90c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1863be0 0x5625f1865fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5625f17bd200 con 0x5625f1859320 2024-09-19T05:58:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5625f1763dc0 con 0x5625f1858a00 2024-09-19T05:58:15.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe92be02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5625f17b91c0 con 0x5625f1863be0 2024-09-19T05:58:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe922ffd640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 0x7fe90c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe9227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe9227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46690/0 (socket says 172.21.15.38:46690) 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe9227fc640 1 -- 172.21.15.38:0/4271270674 learned_addr learned my addr 172.21.15.38:0/4271270674 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.060+0000 7fe928a27640 1 -- 172.21.15.38:0/4102200184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe91c0d01f0 con 0x7fe924106f10 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1858a00 0x5625f1858de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9237fe640 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1863be0 0x5625f1865fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 -- 172.21.15.38:0/4271270674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1863be0 msgr2=0x5625f1865fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1863be0 0x5625f1865fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 -- 172.21.15.38:0/4271270674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1858a00 msgr2=0x5625f1858de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1858a00 0x5625f1858de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 -- 172.21.15.38:0/4271270674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f152e620 con 0x5625f1859320 2024-09-19T05:58:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe922ffd640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 0x7fe90c0a5780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe914005d00 tx=0x7fe91400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe9227fc640 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe918000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c004105e15902a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe909ffb640 1 -- 172.21.15.38:0/4271270674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918035070 con 0x5625f1859320 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe909ffb640 1 -- 172.21.15.38:0/4271270674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918005020 con 0x5625f1859320 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.061+0000 7fe909ffb640 1 -- 172.21.15.38:0/4271270674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180049e0 con 0x5625f1859320 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- 172.21.15.38:0/4271270674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 msgr2=0x5625f18636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe918000bc0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- 172.21.15.38:0/4271270674 shutdown_connections 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1863be0 0x5625f1865fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f18636a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 --2- 172.21.15.38:0/4271270674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1858a00 0x5625f1858de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- 172.21.15.38:0/4271270674 >> 172.21.15.38:0/4271270674 conn(0x5625f1854f90 msgr2=0x5625f1855370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- 172.21.15.38:0/4271270674 shutdown_connections 2024-09-19T05:58:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- 172.21.15.38:0/4271270674 wait complete. 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.062+0000 7fe92be02740 1 -- start start 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 Processor -- start 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1863be0 0x5625f1860cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5625f18667b0 con 0x5625f1859320 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5625f1757c50 con 0x5625f1863be0 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5625f1866930 con 0x5625f1858a00 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe92be02740 10 client.0 _ll_register_callbacks cb 0x5625f1769ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46706/0 (socket says 172.21.15.38:46706) 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 -- 172.21.15.38:0/3790575271 learned_addr learned my addr 172.21.15.38:0/3790575271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1863be0 0x5625f1860cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 -- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 msgr2=0x5625f18634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 -- 172.21.15.38:0/3790575271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1863be0 msgr2=0x5625f1860cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1863be0 0x5625f1860cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 -- 172.21.15.38:0/3790575271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f18614a0 con 0x5625f1859320 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe918031470 tx=0x7fe918031790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5850e794ee1c721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.063+0000 7fe902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918035040 con 0x5625f1859320 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe918004230 con 0x5625f1859320 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe918005020 con 0x5625f1859320 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe918058050 con 0x5625f1859320 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-09-19T05:58:15.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625f18722e0 con 0x5625f1859320 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918005020 con 0x5625f1859320 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe918003850 con 0x5625f1859320 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 10 client.16122 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 20 client.16122 populate_metadata read hostname 'smithi038' 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f1873630 0x5625f1875a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 4 client.16122 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.064+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f17ca770 con 0x5625f1873630 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.065+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f1873630 0x5625f1875a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.065+0000 7fe902ffd640 10 client.16122 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.065+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f1873630 0x5625f1875a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9046599fb83a3f server_cookie=475306459a36b059 in_seq=0 out_seq=0 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe902ffd640 1 -- 172.21.15.38:0/3790575271 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625f17ca770 con 0x5625f1873630 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe902ffd640 10 client.16122 handle_command_reply: tid=0 2024-09-19T05:58:15.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 client.16122 shutdown 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 msgr2=0x5625f1860780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe918031470 tx=0x7fe918031790 comp rx=0 tx=0).stop 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 shutdown_connections 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f1873630 0x5625f1875a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1863be0 0x5625f1860cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1859320 0x5625f1860780 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 --2- 172.21.15.38:0/3790575271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe902ffd640 0 client.16122 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 >> 172.21.15.38:0/3790575271 conn(0x5625f1854f90 msgr2=0x5625f18541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 shutdown_connections 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 1 -- 172.21.15.38:0/3790575271 wait complete. 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.066+0000 7fe92be02740 20 client.16122 trim_cache size 0 max 16384 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 Processor -- start 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 -- start start 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f1854020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f186fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241aeb80 con 0x5625f1863be0 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241b06c0 con 0x5625f1859320 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe92be02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241af250 con 0x5625f1858a00 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.083+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46710/0 (socket says 172.21.15.38:46710) 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 -- 172.21.15.38:0/3466898806 learned_addr learned my addr 172.21.15.38:0/3466898806 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f1854020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f186fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 -- 172.21.15.38:0/3466898806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 msgr2=0x5625f1854020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f1854020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 -- 172.21.15.38:0/3466898806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 msgr2=0x5625f186fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f186fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 -- 172.21.15.38:0/3466898806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f18558d0 con 0x5625f1863be0 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09c3e0 tx=0x7fe91c00af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a768c775025d3fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9097fa640 1 -- 172.21.15.38:0/3466898806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c09c450 con 0x5625f1863be0 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9097fa640 1 -- 172.21.15.38:0/3466898806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0a0800 con 0x5625f1863be0 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.084+0000 7fe9097fa640 1 -- 172.21.15.38:0/3466898806 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c098ac0 con 0x5625f1863be0 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- 172.21.15.38:0/3466898806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 msgr2=0x5625f1878e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe91c09c3e0 tx=0x7fe91c00af30 comp rx=0 tx=0).stop 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- 172.21.15.38:0/3466898806 shutdown_connections 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f1878e30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f186fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 --2- 172.21.15.38:0/3466898806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f1854020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- 172.21.15.38:0/3466898806 >> 172.21.15.38:0/3466898806 conn(0x5625f1854f90 msgr2=0x5625f1862800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- 172.21.15.38:0/3466898806 shutdown_connections 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- 172.21.15.38:0/3466898806 wait complete. 2024-09-19T05:58:15.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 -- start start 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.085+0000 7fe92be02740 1 Processor -- start 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f18d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241afff0 con 0x5625f1863be0 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241af920 con 0x5625f1859320 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9241acb60 con 0x5625f1858a00 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe92be02740 10 client.0 _ll_register_callbacks cb 0x7fe9180315d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46716/0 (socket says 172.21.15.38:46716) 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 -- 172.21.15.38:0/3799226664 learned_addr learned my addr 172.21.15.38:0/3799226664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f18d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 -- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 msgr2=0x5625f18d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 -- 172.21.15.38:0/3799226664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 msgr2=0x5625f18d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f18d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 -- 172.21.15.38:0/3799226664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f18d9d00 con 0x5625f1863be0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9227fc640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f18d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe9237fe640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00ab20 tx=0x7fe91c0a5150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdc1f9639cd4277 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.086+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f8e0 con 0x5625f1863be0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fe91c00fc80 con 0x5625f1863be0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c0a0800 con 0x5625f1863be0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe91c0a4c30 con 0x5625f1863be0 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-09-19T05:58:15.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625f18e2270 con 0x5625f1863be0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0a0800 con 0x5625f1863be0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe91c09b930 con 0x5625f1863be0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 10 client.16128 resolve_mds: validated gid 24496 aka daemon mds.i 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 20 client.16128 populate_metadata read hostname 'smithi038' 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f18e28e0 0x5625f18e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 4 client.16128 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.087+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 --> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5625f17c4c80 con 0x5625f18e28e0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f18e28e0 0x5625f18e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe922ffd640 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f18e28e0 0x5625f18e4cd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466712a31857a4e server_cookie=a9559c29c1ab1b61 in_seq=0 out_seq=0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe902ffd640 10 client.16128 ms_handle_connect on v2:172.21.15.179:6838/2081935059 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe902ffd640 1 -- 172.21.15.38:0/3799226664 <== mds.0 v2:172.21.15.179:6838/2081935059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5625f17c4c80 con 0x5625f18e28e0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe902ffd640 10 client.16128 handle_command_reply: tid=0 2024-09-19T05:58:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe92be02740 1 client.16128 shutdown 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 msgr2=0x5625f18d9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.088+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00ab20 tx=0x7fe91c0a5150 comp rx=0 tx=0).stop 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 shutdown_connections 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:6838/2081935059,v1:172.21.15.179:6839/2081935059] conn(0x5625f18e28e0 0x5625f18e4cd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 28, 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24496, 2024-09-19T05:58:15.224 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 56, 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": {} 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f1863be0 0x5625f18d9660 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625f1859320 0x5625f18d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 --2- 172.21.15.38:0/3799226664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625f1858a00 0x5625f18d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 >> 172.21.15.38:0/3799226664 conn(0x5625f1854f90 msgr2=0x5625f185a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe902ffd640 0 client.16128 ms_handle_reset on v2:172.21.15.179:6838/2081935059 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 shutdown_connections 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 1 -- 172.21.15.38:0/3799226664 wait complete. 2024-09-19T05:58:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe92be02740 20 client.16128 trim_cache size 0 max 16384 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 msgr2=0x7fe90c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 0x7fe90c0a5780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe914005d00 tx=0x7fe91400b410 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 msgr2=0x7fe92419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00cce0 tx=0x7fe91c00a620 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 shutdown_connections 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fe90c0a32c0 0x7fe90c0a5780 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe924106f10 0x7fe92419cce0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe924102790 0x7fe924198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 --2- 172.21.15.38:0/4102200184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe924101de0 0x7fe924198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 >> 172.21.15.38:0/4102200184 conn(0x7fe9240fd8c0 msgr2=0x7fe9240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 shutdown_connections 2024-09-19T05:58:15.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:15.089+0000 7fe929a29640 1 -- 172.21.15.38:0/4102200184 wait complete. 2024-09-19T05:58:15.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:15 smithi179 ceph-mon[35781]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:58:15.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:15 smithi047 ceph-mon[35327]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:58:15.891 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:15 smithi038 ceph-mon[31227]: pgmap v863: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-19T05:58:15.922 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T05:58:15.922 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T05:58:15.932 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:58:16.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 -- 172.21.15.38:0/1962965835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 msgr2=0x7f13fc100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 --2- 172.21.15.38:0/1962965835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc100e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009900 tx=0x7f13ec02a410 comp rx=0 tx=0).stop 2024-09-19T05:58:16.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 -- 172.21.15.38:0/1962965835 shutdown_connections 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 --2- 172.21.15.38:0/1962965835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc1025e0 0x7f13fc108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 --2- 172.21.15.38:0/1962965835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc101c20 0x7f13fc1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 --2- 172.21.15.38:0/1962965835 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc100e30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 -- 172.21.15.38:0/1962965835 >> 172.21.15.38:0/1962965835 conn(0x7f13fc0fc190 msgr2=0x7f13fc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 -- 172.21.15.38:0/1962965835 shutdown_connections 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.040+0000 7f1402d31640 1 -- 172.21.15.38:0/1962965835 wait complete. 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 Processor -- start 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 -- start start 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc101c20 0x7f13fc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc1025e0 0x7f13fc19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10c6b0 con 0x7f13fc100a30 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10c530 con 0x7f13fc1025e0 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1402d31640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc10c830 con 0x7f13fc101c20 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46742/0 (socket says 172.21.15.38:46742) 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 -- 172.21.15.38:0/260807408 learned_addr learned my addr 172.21.15.38:0/260807408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f14012ce640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc1025e0 0x7f13fc19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 -- 172.21.15.38:0/260807408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc101c20 msgr2=0x7f13fc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f13fbfff640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc101c20 0x7f13fc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc101c20 0x7f13fc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 -- 172.21.15.38:0/260807408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc1025e0 msgr2=0x7f13fc19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc1025e0 0x7f13fc19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.041+0000 7f1400acd640 1 -- 172.21.15.38:0/260807408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc19fb70 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f1400acd640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f13ec006010 tx=0x7f13ec031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec03f030 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc19fda0 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec0044a0 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec035070 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.042+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1a01d0 con 0x7f13fc100a30 2024-09-19T05:58:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.043+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc107e90 con 0x7f13fc100a30 2024-09-19T05:58:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.044+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f13ec042030 con 0x7f13fc100a30 2024-09-19T05:58:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.045+0000 7f13f9ffb640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 0x7f13e80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.045+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f13ec0f02b0 con 0x7f13fc100a30 2024-09-19T05:58:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.045+0000 7f13fbfff640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 0x7f13e80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.045+0000 7f13fbfff640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 0x7f13e80a5710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005fd0 tx=0x7f13e4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.049+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13ec0bcbb0 con 0x7f13fc100a30 2024-09-19T05:58:16.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.225+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13fc062fb0 con 0x7f13fc100a30 2024-09-19T05:58:16.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.226+0000 7f13f9ffb640 1 -- 172.21.15.38:0/260807408 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24875 (secure 0 0 0) 0x7f13ec0bc4b0 con 0x7f13fc100a30 2024-09-19T05:58:16.228 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:16.229 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:41:11.190669+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 msgr2=0x7f13e80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 0x7f13e80a5710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005fd0 tx=0x7f13e4005850 comp rx=0 tx=0).stop 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 msgr2=0x7f13fc06af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f13ec006010 tx=0x7f13ec031b60 comp rx=0 tx=0).stop 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 shutdown_connections 2024-09-19T05:58:16.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f13e80a3250 0x7f13e80a5710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13fc1025e0 0x7f13fc19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13fc101c20 0x7f13fc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 --2- 172.21.15.38:0/260807408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13fc100a30 0x7f13fc06af30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 >> 172.21.15.38:0/260807408 conn(0x7f13fc0fc190 msgr2=0x7f13fc105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 shutdown_connections 2024-09-19T05:58:16.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.229+0000 7f1402d31640 1 -- 172.21.15.38:0/260807408 wait complete. 2024-09-19T05:58:16.242 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:34:23.998201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:38:58.787912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:39:00.870380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-19T05:39:07.367942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-19T05:58:16.243 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T05:58:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- 172.21.15.38:0/4104261201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 msgr2=0x7f221c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- 172.21.15.38:0/4104261201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c102070 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f220c009900 tx=0x7f220c02a410 comp rx=0 tx=0).stop 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- 172.21.15.38:0/4104261201 shutdown_connections 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- 172.21.15.38:0/4104261201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- 172.21.15.38:0/4104261201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c102070 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- 172.21.15.38:0/4104261201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 0x7f221c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- 172.21.15.38:0/4104261201 >> 172.21.15.38:0/4104261201 conn(0x7f221c0fc140 msgr2=0x7f221c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- 172.21.15.38:0/4104261201 shutdown_connections 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- 172.21.15.38:0/4104261201 wait complete. 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 Processor -- start 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 -- start start 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 0x7f221c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.348+0000 7f2223c2e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10c680 con 0x7f221c101bf0 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10c500 con 0x7f221c1025b0 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f221c10c800 con 0x7f221c100a00 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46754/0 (socket says 172.21.15.38:46754) 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 -- 172.21.15.38:0/1809298975 learned_addr learned my addr 172.21.15.38:0/1809298975 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22219ca640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 0x7f221c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22221cb640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 -- 172.21.15.38:0/1809298975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 msgr2=0x7f221c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 0x7f221c069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 -- 172.21.15.38:0/1809298975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 msgr2=0x7f221c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 -- 172.21.15.38:0/1809298975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c19faf0 con 0x7f221c101bf0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22221cb640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f22211c9640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f220c009230 tx=0x7f220c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c03f030 con 0x7f221c101bf0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c003ce0 con 0x7f221c101bf0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c19fd20 con 0x7f221c101bf0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c035070 con 0x7f221c101bf0 2024-09-19T05:58:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.349+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c1a01c0 con 0x7f221c101bf0 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.350+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c107f00 con 0x7f221c101bf0 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.352+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f220c042030 con 0x7f221c101bf0 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.353+0000 7f2212ffd640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 0x7f22080a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.353+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f220c0f06a0 con 0x7f221c101bf0 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.353+0000 7f22219ca640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 0x7f22080a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.354+0000 7f22219ca640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 0x7f22080a5710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f221800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.356+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f220c0bd000 con 0x7f221c101bf0 2024-09-19T05:58:16.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.536+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f221c02d970 con 0x7f221c101bf0 2024-09-19T05:58:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.536+0000 7f2212ffd640 1 -- 172.21.15.38:0/1809298975 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f220c0bc900 con 0x7f221c101bf0 2024-09-19T05:58:16.538 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-19T05:58:16.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 msgr2=0x7f22080a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 0x7f22080a5710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f221800b390 comp rx=0 tx=0).stop 2024-09-19T05:58:16.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 msgr2=0x7f221c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f220c009230 tx=0x7f220c004360 comp rx=0 tx=0).stop 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 shutdown_connections 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f22080a3250 0x7f22080a5710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f221c1025b0 0x7f221c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f221c101bf0 0x7f221c19cbc0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 --2- 172.21.15.38:0/1809298975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f221c100a00 0x7f221c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.538+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 >> 172.21.15.38:0/1809298975 conn(0x7f221c0fc140 msgr2=0x7f221c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.539+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 shutdown_connections 2024-09-19T05:58:16.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.539+0000 7f2223c2e640 1 -- 172.21.15.38:0/1809298975 wait complete. 2024-09-19T05:58:16.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:16 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/260807408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:16.551 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 -- 172.21.15.38:0/241613779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 msgr2=0x7fee301020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 --2- 172.21.15.38:0/241613779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee301020a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fee24009900 tx=0x7fee2402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 -- 172.21.15.38:0/241613779 shutdown_connections 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 --2- 172.21.15.38:0/241613779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee30108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 --2- 172.21.15.38:0/241613779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee301020a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 --2- 172.21.15.38:0/241613779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 0x7fee30100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 -- 172.21.15.38:0/241613779 >> 172.21.15.38:0/241613779 conn(0x7fee300fc1b0 msgr2=0x7fee300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 -- 172.21.15.38:0/241613779 shutdown_connections 2024-09-19T05:58:16.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.656+0000 7fee363fd640 1 -- 172.21.15.38:0/241613779 wait complete. 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 Processor -- start 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 -- start start 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 0x7fee30197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee3019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fee3010c6b0 con 0x7fee30101c20 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee3010c530 con 0x7fee301025e0 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee363fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee3010c830 con 0x7fee30100a30 2024-09-19T05:58:16.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46766/0 (socket says 172.21.15.38:46766) 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 -- 172.21.15.38:0/3376809021 learned_addr learned my addr 172.21.15.38:0/3376809021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee3499a640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee3019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2ffff640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 0x7fee30197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 -- 172.21.15.38:0/3376809021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 msgr2=0x7fee30197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 0x7fee30197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 -- 172.21.15.38:0/3376809021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 msgr2=0x7fee3019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee3019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee2f7fe640 1 -- 172.21.15.38:0/3376809021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3019fb60 con 0x7fee30101c20 2024-09-19T05:58:16.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.657+0000 7fee3499a640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee3019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:58:16.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee2f7fe640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fee24005940 tx=0x7fee24004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2403f030 con 0x7fee30101c20 2024-09-19T05:58:16.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3019fdf0 con 0x7fee30101c20 2024-09-19T05:58:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee24003ce0 con 0x7fee30101c20 2024-09-19T05:58:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24035070 con 0x7fee30101c20 2024-09-19T05:58:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.658+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a0250 con 0x7fee30101c20 2024-09-19T05:58:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.659+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee301a0480 con 0x7fee30101c20 2024-09-19T05:58:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.660+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fee24042030 con 0x7fee30101c20 2024-09-19T05:58:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.661+0000 7fee2d7fa640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 0x7fee180a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.661+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7fee240f0730 con 0x7fee30101c20 2024-09-19T05:58:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.661+0000 7fee2ffff640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 0x7fee180a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.661+0000 7fee2ffff640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 0x7fee180a5710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.665+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee240bd090 con 0x7fee30101c20 2024-09-19T05:58:16.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:16 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/260807408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:16.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:16 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/260807408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.842+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fee301a0810 con 0x7fee30101c20 2024-09-19T05:58:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.843+0000 7fee2d7fa640 1 -- 172.21.15.38:0/3376809021 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fee240bc990 con 0x7fee30101c20 2024-09-19T05:58:16.845 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-19T05:58:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 msgr2=0x7fee180a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 0x7fee180a5710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee20005fd0 tx=0x7fee20005850 comp rx=0 tx=0).stop 2024-09-19T05:58:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 msgr2=0x7fee30198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fee24005940 tx=0x7fee24004360 comp rx=0 tx=0).stop 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 shutdown_connections 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fee180a3250 0x7fee180a5710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee301025e0 0x7fee3019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee30101c20 0x7fee30198480 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 --2- 172.21.15.38:0/3376809021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee30100a30 0x7fee30197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 >> 172.21.15.38:0/3376809021 conn(0x7fee300fc1b0 msgr2=0x7fee3006a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 shutdown_connections 2024-09-19T05:58:16.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.845+0000 7fee363fd640 1 -- 172.21.15.38:0/3376809021 wait complete. 2024-09-19T05:58:16.857 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T05:58:16.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- 172.21.15.38:0/993097667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c1025e0 msgr2=0x7f7f3c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/993097667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c1025e0 0x7f7f3c108e70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009900 tx=0x7f7f3402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:16.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- 172.21.15.38:0/993097667 shutdown_connections 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/993097667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c1025e0 0x7f7f3c108e70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/993097667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c101c20 0x7f7f3c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/993097667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3c100a30 0x7f7f3c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- 172.21.15.38:0/993097667 >> 172.21.15.38:0/993097667 conn(0x7f7f3c0fc1b0 msgr2=0x7f7f3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- 172.21.15.38:0/993097667 shutdown_connections 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- 172.21.15.38:0/993097667 wait complete. 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 Processor -- start 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 -- start start 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c100a30 0x7f7f3c1049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.963+0000 7f7f42ff6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3c1025e0 0x7f7f3c103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f42ff6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c10c730 con 0x7f7f3c101c20 2024-09-19T05:58:16.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f42ff6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c10c5b0 con 0x7f7f3c100a30 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f42ff6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c10c8b0 con 0x7f7f3c1025e0 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46784/0 (socket says 172.21.15.38:46784) 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 -- 172.21.15.38:0/3912289209 learned_addr learned my addr 172.21.15.38:0/3912289209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f40d92640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c100a30 0x7f7f3c1049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 -- 172.21.15.38:0/3912289209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3c1025e0 msgr2=0x7f7f3c103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3c1025e0 0x7f7f3c103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 -- 172.21.15.38:0/3912289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c100a30 msgr2=0x7f7f3c1049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c100a30 0x7f7f3c1049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 -- 172.21.15.38:0/3912289209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c103dd0 con 0x7f7f3c101c20 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f3bfff640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00ebc0 tx=0x7f7f2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c010430 con 0x7f7f3c101c20 2024-09-19T05:58:16.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c1a1fe0 con 0x7f7f3c101c20 2024-09-19T05:58:16.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c010ab0 con 0x7f7f3c101c20 2024-09-19T05:58:16.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c018b30 con 0x7f7f3c101c20 2024-09-19T05:58:16.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.964+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c1a2570 con 0x7f7f3c101c20 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.965+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c1a2890 con 0x7f7f3c101c20 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.967+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7f2c018cd0 con 0x7f7f3c101c20 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.967+0000 7f7f39ffb640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 0x7f7f280a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.968+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f7f2c0d0650 con 0x7f7f3c101c20 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.968+0000 7f7f40d92640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 0x7f7f280a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.968+0000 7f7f40d92640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 0x7f7f280a5710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c101a80 tx=0x7f7f24009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:16.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:16.971+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f2c09cfb0 con 0x7f7f3c101c20 2024-09-19T05:58:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.146+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f3c1a2b80 con 0x7f7f3c101c20 2024-09-19T05:58:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.147+0000 7f7f39ffb640 1 -- 172.21.15.38:0/3912289209 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f7f2c09c8b0 con 0x7f7f3c101c20 2024-09-19T05:58:17.149 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 msgr2=0x7f7f280a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 0x7f7f280a5710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c101a80 tx=0x7f7f24009110 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 msgr2=0x7f7f3c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00ebc0 tx=0x7f7f2c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 shutdown_connections 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f7f280a3250 0x7f7f280a5710 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f3c1025e0 0x7f7f3c103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f3c101c20 0x7f7f3c103000 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 --2- 172.21.15.38:0/3912289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f3c100a30 0x7f7f3c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 >> 172.21.15.38:0/3912289209 conn(0x7f7f3c0fc1b0 msgr2=0x7f7f3c06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 shutdown_connections 2024-09-19T05:58:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.149+0000 7f7f42ff6640 1 -- 172.21.15.38:0/3912289209 wait complete. 2024-09-19T05:58:17.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-19T05:58:17.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 -- 172.21.15.38:0/2656205950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b78100a30 msgr2=0x7f4b78100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/2656205950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b78100a30 0x7f4b78100e30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009900 tx=0x7f4b7402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 -- 172.21.15.38:0/2656205950 shutdown_connections 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/2656205950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781025e0 0x7f4b78108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/2656205950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78101c20 0x7f4b781020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/2656205950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b78100a30 0x7f4b78100e30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 -- 172.21.15.38:0/2656205950 >> 172.21.15.38:0/2656205950 conn(0x7f4b780fc1b0 msgr2=0x7f4b780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 -- 172.21.15.38:0/2656205950 shutdown_connections 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.267+0000 7f4b7fd86640 1 -- 172.21.15.38:0/2656205950 wait complete. 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 Processor -- start 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 -- start start 2024-09-19T05:58:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78100a30 0x7f4b7810b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101c20 0x7f4b781099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d820 con 0x7f4b781025e0 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d6a0 con 0x7f4b78100a30 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7fd86640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d9a0 con 0x7f4b78101c20 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46804/0 (socket says 172.21.15.38:46804) 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 -- 172.21.15.38:0/3481654562 learned_addr learned my addr 172.21.15.38:0/3481654562 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7db22640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78100a30 0x7f4b7810b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7d321640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101c20 0x7f4b781099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 -- 172.21.15.38:0/3481654562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101c20 msgr2=0x7f4b781099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101c20 0x7f4b781099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 -- 172.21.15.38:0/3481654562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78100a30 msgr2=0x7f4b7810b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78100a30 0x7f4b7810b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.268+0000 7f4b7e323640 1 -- 172.21.15.38:0/3481654562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7810a7c0 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b7e323640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b970 tx=0x7f4b7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70019070 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7000cb40 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b781a2080 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70007650 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.269+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781a2650 con 0x7f4b781025e0 2024-09-19T05:58:17.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.270+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b78107f10 con 0x7f4b781025e0 2024-09-19T05:58:17.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.271+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4b7001c070 con 0x7f4b781025e0 2024-09-19T05:58:17.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.272+0000 7f4b6effd640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 0x7f4b640a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.272+0000 7f4b7db22640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 0x7f4b640a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.272+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f4b700cf740 con 0x7f4b781025e0 2024-09-19T05:58:17.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.273+0000 7f4b7db22640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 0x7f4b640a5710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009230 tx=0x7f4b740315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.275+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b7009ba50 con 0x7f4b781025e0 2024-09-19T05:58:17.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.450+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4b78062df0 con 0x7f4b781025e0 2024-09-19T05:58:17.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.451+0000 7f4b6effd640 1 -- 172.21.15.38:0/3481654562 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f4b7009fd50 con 0x7f4b781025e0 2024-09-19T05:58:17.452 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-19T05:58:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.452+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 msgr2=0x7f4b640a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.452+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 0x7f4b640a5710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009230 tx=0x7f4b740315b0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.452+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 msgr2=0x7f4b78109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.452+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b970 tx=0x7f4b7000be30 comp rx=0 tx=0).stop 2024-09-19T05:58:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 shutdown_connections 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f4b640a3250 0x7f4b640a5710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b781025e0 0x7f4b78109f30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101c20 0x7f4b781099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 --2- 172.21.15.38:0/3481654562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78100a30 0x7f4b7810b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 >> 172.21.15.38:0/3481654562 conn(0x7f4b780fc1b0 msgr2=0x7f4b781050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 shutdown_connections 2024-09-19T05:58:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.453+0000 7f4b7fd86640 1 -- 172.21.15.38:0/3481654562 wait complete. 2024-09-19T05:58:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:17 smithi038 ceph-mon[31227]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1809298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3376809021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:17 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3912289209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 -- 172.21.15.38:0/1600502741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 msgr2=0x7feb6c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 --2- 172.21.15.38:0/1600502741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c104980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb68009900 tx=0x7feb6802a410 comp rx=0 tx=0).stop 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 -- 172.21.15.38:0/1600502741 shutdown_connections 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 --2- 172.21.15.38:0/1600502741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 0x7feb6c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 --2- 172.21.15.38:0/1600502741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 --2- 172.21.15.38:0/1600502741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c104980 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 -- 172.21.15.38:0/1600502741 >> 172.21.15.38:0/1600502741 conn(0x7feb6c0fc170 msgr2=0x7feb6c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:17.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 -- 172.21.15.38:0/1600502741 shutdown_connections 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.568+0000 7feb73665640 1 -- 172.21.15.38:0/1600502741 wait complete. 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 Processor -- start 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 -- start start 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 0x7feb6c100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10d7f0 con 0x7feb6c102590 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10d670 con 0x7feb6c1077f0 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb73665640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10d970 con 0x7feb6c104ec0 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46370/0 (socket says 172.21.15.38:46370) 2024-09-19T05:58:17.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 -- 172.21.15.38:0/2436005332 learned_addr learned my addr 172.21.15.38:0/2436005332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71c02640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 0x7feb6c100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb70c00640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 -- 172.21.15.38:0/2436005332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 msgr2=0x7feb6c100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 -- 172.21.15.38:0/2436005332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 msgr2=0x7feb6c100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 0x7feb6c100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb71401640 1 -- 172.21.15.38:0/2436005332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c101310 con 0x7feb6c102590 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.569+0000 7feb70c00640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb71401640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb68009230 tx=0x7feb680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6803f030 con 0x7feb6c102590 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68003ce0 con 0x7feb6c102590 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a1f70 con 0x7feb6c102590 2024-09-19T05:58:17.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68035070 con 0x7feb6c102590 2024-09-19T05:58:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.570+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a2380 con 0x7feb6c102590 2024-09-19T05:58:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.571+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c0fdc60 con 0x7feb6c102590 2024-09-19T05:58:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.572+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7feb68042030 con 0x7feb6c102590 2024-09-19T05:58:17.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.573+0000 7feb627fc640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 0x7feb580a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:17.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.573+0000 7feb70c00640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 0x7feb580a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:17.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.573+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7feb680f01c0 con 0x7feb6c102590 2024-09-19T05:58:17.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.574+0000 7feb70c00640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 0x7feb580a5710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb6c101c70 tx=0x7feb5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:17.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.577+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb680bcb20 con 0x7feb6c102590 2024-09-19T05:58:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:17 smithi179 ceph-mon[35781]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:17 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1809298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:17 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3376809021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:17 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3912289209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:17 smithi047 ceph-mon[35327]: pgmap v864: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T05:58:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:17 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1809298975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:17 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3376809021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:17 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3912289209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:58:17.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.801+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7feb6c062f30 con 0x7feb6c102590 2024-09-19T05:58:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.810+0000 7feb627fc640 1 -- 172.21.15.38:0/2436005332 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742341 (secure 0 0 0) 0x7feb680bc420 con 0x7feb6c102590 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 18, 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "active_gid": 14150, 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "active_name": "x", 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addrs": { 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-19T05:58:17.822 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6800", 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4137587885 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6801", 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 4137587885 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addr": "172.21.15.38:6801/4137587885", 2024-09-19T05:58:17.823 INFO:teuthology.orchestra.run.smithi038.stdout: "active_change": "2024-09-19T05:32:09.496858+0000", 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "available": true, 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "standbys": [ 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24109, 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "z", 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:17.824 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.826 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.827 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:17.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.829 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-19T05:58:17.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-19T05:58:17.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.836 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-19T05:58:17.839 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.840 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-19T05:58:17.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-19T05:58:17.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.843 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:58:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.846 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:17.848 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.849 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.850 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.852 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-19T05:58:17.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:17.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:17.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:17.859 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.864 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:58:17.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:58:17.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.869 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-19T05:58:17.871 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.872 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.875 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-19T05:58:17.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.879 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:58:17.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:58:17.882 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.884 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-19T05:58:17.886 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-19T05:58:17.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.888 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.889 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.892 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:58:17.893 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.894 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.895 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-19T05:58:17.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-19T05:58:17.901 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-19T05:58:17.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-19T05:58:17.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.904 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.905 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.906 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.912 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.914 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.916 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-19T05:58:17.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-19T05:58:17.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.919 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.920 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.921 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:58:17.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.926 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.927 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-19T05:58:17.932 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.933 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.934 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:17.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.936 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.939 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:17.940 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.941 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.943 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.946 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.947 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.948 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.949 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.950 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:58:17.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:58:17.952 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.953 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.954 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.955 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.956 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.958 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.959 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.960 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.961 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.963 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.965 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.966 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.967 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.970 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:58:17.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:58:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.973 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.975 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.978 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.979 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.981 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:58:17.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:58:17.983 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:58:17.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.985 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.988 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.989 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.990 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.991 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.993 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.994 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:17.995 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:17.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.997 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:17.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.000 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.001 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:18.003 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.004 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.005 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.006 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.007 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.008 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.009 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.010 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.011 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.012 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.013 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.014 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.015 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.016 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.017 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:58:18.019 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.022 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.023 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.026 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.028 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.029 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.030 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.031 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-19T05:58:18.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.033 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.034 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.035 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.038 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.039 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.041 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.042 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:58:18.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:58:18.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.046 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.047 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.050 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.056 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.057 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-19T05:58:18.058 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.060 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.062 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.063 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.065 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.066 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.069 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-19T05:58:18.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.077 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.079 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.080 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.081 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.082 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.083 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.084 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-19T05:58:18.086 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.087 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.090 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.091 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.092 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.093 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.097 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-19T05:58:18.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.100 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.105 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.107 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.108 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.109 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.110 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.112 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.113 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.115 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.116 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.122 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.123 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.125 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.127 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.129 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.131 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.132 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-19T05:58:18.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.134 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.137 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.140 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.142 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.146 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-19T05:58:18.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.150 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.153 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.156 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.159 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-19T05:58:18.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.161 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.165 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.168 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.170 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.172 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.173 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.174 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.176 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.179 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.180 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.182 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.183 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-19T05:58:18.186 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.187 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.192 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-19T05:58:18.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.194 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-19T05:58:18.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.197 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.198 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.200 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.201 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.202 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.203 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.204 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.205 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.206 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.208 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.209 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.212 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.213 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.214 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.216 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.217 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.218 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.219 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.221 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.222 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:58:18.225 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.226 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.228 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.230 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.233 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.235 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.238 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-19T05:58:18.240 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:18.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.242 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-19T05:58:18.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.246 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.247 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.248 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.250 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.251 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.252 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.253 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.255 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.256 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.258 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.259 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.262 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.263 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-19T05:58:18.266 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.269 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.272 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.275 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.276 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.281 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24113, 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "y", 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-19T05:58:18.283 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.284 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.285 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.286 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.288 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.289 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.290 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.292 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-19T05:58:18.293 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-19T05:58:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-19T05:58:18.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.296 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.298 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.300 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-19T05:58:18.301 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-19T05:58:18.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-19T05:58:18.303 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.305 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:18.307 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.308 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.309 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.311 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.312 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.313 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.314 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.315 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:58:18.317 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:58:18.318 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:58:18.319 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-19T05:58:18.320 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-19T05:58:18.321 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-19T05:58:18.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.324 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.325 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.326 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-19T05:58:18.327 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.328 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.329 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.330 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.331 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-19T05:58:18.332 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:58:18.333 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-19T05:58:18.334 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-19T05:58:18.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.337 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.338 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.339 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-19T05:58:18.340 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.341 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.342 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.344 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:58:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.346 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.347 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.348 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:58:18.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:58:18.350 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.352 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.353 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.354 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:58:18.355 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.356 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.358 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.359 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:58:18.363 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.364 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.365 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:58:18.366 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.367 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.368 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.369 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.370 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.371 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-19T05:58:18.372 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.373 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:58:18.374 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.375 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.376 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.377 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.378 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:58:18.379 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-19T05:58:18.380 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.381 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.382 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.384 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.385 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.386 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.387 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.388 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.389 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.390 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.391 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.392 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.393 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.394 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.395 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.396 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.397 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:58:18.398 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:58:18.399 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:58:18.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:58:18.401 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.402 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.403 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.404 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.405 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.406 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.407 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.408 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.409 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.410 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.411 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.412 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.413 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.414 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.415 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:58:18.416 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:58:18.417 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:58:18.418 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.419 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.420 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.421 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.422 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.423 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.424 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.425 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.426 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.427 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.428 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.429 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:58:18.430 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.431 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.432 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.433 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.434 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.435 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:58:18.436 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:58:18.437 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:58:18.438 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.439 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.442 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.443 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:58:18.444 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.445 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.446 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.447 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.448 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.449 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.450 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.451 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.452 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.453 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.454 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.455 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.456 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.457 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:18.458 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.459 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.460 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.461 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-19T05:58:18.462 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.463 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.464 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.465 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.466 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.467 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.468 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:58:18.469 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.470 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.471 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.472 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.473 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.474 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-19T05:58:18.475 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.476 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.477 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.480 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.481 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.482 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.483 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.484 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.485 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.487 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.488 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-19T05:58:18.490 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.491 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.492 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.493 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.494 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.495 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.496 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.497 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.498 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-19T05:58:18.499 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.500 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.501 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.502 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-19T05:58:18.503 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.504 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.506 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.507 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.508 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.509 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.510 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.511 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.512 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.513 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.514 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.515 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.516 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.517 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.518 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.519 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.520 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.522 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.523 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.524 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.525 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.526 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.527 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.528 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.529 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.530 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.531 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-19T05:58:18.532 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.533 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.534 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:58:18.535 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.536 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.537 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.538 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.539 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.540 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.541 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.543 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.544 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.545 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.546 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.547 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.548 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.549 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.550 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.551 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.552 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.554 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.555 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.556 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.557 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.559 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.560 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.561 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.562 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.563 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.564 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.565 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.566 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.567 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.568 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.569 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.571 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.572 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.573 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.574 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.575 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.576 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.577 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.578 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.579 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.580 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-19T05:58:18.581 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.582 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.583 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.584 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.586 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.587 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.589 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:58:18.590 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.591 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-19T05:58:18.592 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.593 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.595 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.596 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.597 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.598 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.599 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.601 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.602 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.603 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.604 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-19T05:58:18.605 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.606 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:18.607 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.608 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.609 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.610 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.611 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.612 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.613 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.614 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.615 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.616 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-19T05:58:18.617 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.618 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.619 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.620 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.621 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.622 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.623 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.624 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.626 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.627 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.628 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.629 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.630 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.631 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.632 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:58:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.634 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.635 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.636 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.637 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.638 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.639 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.640 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.641 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.642 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.646 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.647 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.648 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-19T05:58:18.649 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-19T05:58:18.650 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-19T05:58:18.651 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.652 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.653 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.654 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.655 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.656 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.658 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.659 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.660 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.661 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.662 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.663 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.664 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.665 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.666 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.667 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.668 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.669 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.670 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-19T05:58:18.672 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.673 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.674 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.675 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.676 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.677 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.678 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.679 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.680 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.681 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.683 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.684 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.685 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.686 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-19T05:58:18.688 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-19T05:58:18.689 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-19T05:58:18.690 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.691 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.692 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.693 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.694 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.695 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.696 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.697 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.698 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.699 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.700 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.701 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.702 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.703 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.704 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.705 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.706 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.707 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:58:18.709 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.710 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.711 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.712 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.713 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.714 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.715 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.717 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.719 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.720 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.724 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.725 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.726 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.727 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.728 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.729 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.730 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.731 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.732 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.733 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.734 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-19T05:58:18.735 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.736 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.737 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "modules": [ 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard", 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "iostat", 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "nfs", 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: "restful" 2024-09-19T05:58:18.738 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.741 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.743 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.744 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-19T05:58:18.746 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-19T05:58:18.747 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-19T05:58:18.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-19T05:58:18.749 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-19T05:58:18.750 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.751 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.752 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.753 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.755 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.756 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.757 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.759 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-19T05:58:18.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-19T05:58:18.761 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.762 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.765 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.766 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.768 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-19T05:58:18.769 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.770 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.772 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-19T05:58:18.774 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:58:18.775 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.776 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.777 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.780 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.781 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.782 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.783 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-19T05:58:18.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-19T05:58:18.785 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.786 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.787 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.789 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.790 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.791 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:58:18.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-19T05:58:18.793 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.794 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.796 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.797 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:58:18.800 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.802 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.803 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:58:18.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.805 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.806 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.807 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.809 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:58:18.810 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.811 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.812 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.813 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.816 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.817 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.818 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:58:18.820 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:58:18.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-19T05:58:18.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.826 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.831 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.832 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.834 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-19T05:58:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-19T05:58:18.838 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-19T05:58:18.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.840 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.841 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.845 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-19T05:58:18.847 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-19T05:58:18.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.850 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.852 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.853 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.857 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.859 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:58:18.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:58:18.865 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.866 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.867 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.871 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.874 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.882 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:58:18.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:58:18.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.885 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.886 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.890 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.893 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.897 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.898 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.899 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.900 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.902 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.904 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.905 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.906 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-19T05:58:18.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.909 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.911 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.916 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.917 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-19T05:58:18.918 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.919 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.921 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.923 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.924 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-19T05:58:18.927 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-19T05:58:18.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.929 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.930 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.931 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.932 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.936 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.937 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-19T05:58:18.941 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-19T05:58:18.942 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.943 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.944 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-19T05:58:18.946 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.948 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.950 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.951 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.952 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.953 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-19T05:58:18.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.955 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.956 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:58:18.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.959 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.960 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.961 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.962 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.963 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.968 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.970 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.971 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:58:18.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.973 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.974 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.975 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.976 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.977 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-19T05:58:18.978 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.979 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.980 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.981 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.982 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:58:18.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.989 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.990 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.992 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.993 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.994 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:18.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.996 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:18.997 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:18.998 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:18.999 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.000 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.001 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.002 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.004 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.005 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-19T05:58:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.008 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.009 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.010 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.011 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.012 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.013 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-19T05:58:19.014 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.015 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.016 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.019 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.021 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.022 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.023 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.026 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.029 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.030 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.032 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.033 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.035 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.038 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.041 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-19T05:58:19.045 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:58:19.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.048 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.052 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.054 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.056 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.058 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.061 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.062 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.063 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.064 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.065 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.066 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:58:19.068 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-19T05:58:19.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.075 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-19T05:58:19.082 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.083 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.085 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.086 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.087 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.088 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.089 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.090 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.091 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.092 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.094 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.096 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.097 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.098 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.100 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.102 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.105 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.106 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.107 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.108 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.110 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.112 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-19T05:58:19.114 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-19T05:58:19.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.117 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.118 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.119 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.121 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.122 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.125 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.126 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.127 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.128 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.129 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.132 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.133 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.136 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.137 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.141 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:58:19.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.144 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.146 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.148 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.149 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.153 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.154 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.156 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.157 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.158 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.159 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.162 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.164 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.165 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.166 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.169 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.173 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-19T05:58:19.175 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-19T05:58:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.177 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-19T05:58:19.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.185 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-19T05:58:19.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.191 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "services": { 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard": "https://172.21.15.38:8443/" 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "always_on_modules": { 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "octopus": [ 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-19T05:58:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "pacific": [ 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-19T05:58:19.193 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "quincy": [ 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "reef": [ 2024-09-19T05:58:19.194 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "squid": [ 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-19T05:58:19.195 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "last_failure_osd_epoch": 3, 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "active_clients": [ 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3983298487 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "libcephsqlite", 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1979840411 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2725852666 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.198 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2539325944 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-19T05:58:19.199 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3481654562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2436005332' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:18 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3481654562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2436005332' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:19.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:18 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 msgr2=0x7feb580a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 0x7feb580a5710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb6c101c70 tx=0x7feb5c005f90 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 msgr2=0x7feb6c101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb68009230 tx=0x7feb680042f0 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 shutdown_connections 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb580a3250 0x7feb580a5710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb6c1077f0 0x7feb6c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb6c104ec0 0x7feb6c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 --2- 172.21.15.38:0/2436005332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb6c102590 0x7feb6c101f00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 >> 172.21.15.38:0/2436005332 conn(0x7feb6c0fc170 msgr2=0x7feb6c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:19.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 shutdown_connections 2024-09-19T05:58:19.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:17.816+0000 7feb73665640 1 -- 172.21.15.38:0/2436005332 wait complete. 2024-09-19T05:58:19.267 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T05:58:19.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T05:58:19.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3481654562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T05:58:19.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:58:19.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2436005332' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:58:19.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:58:19.269 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:18 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' 2024-09-19T05:58:19.541 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:19 smithi038 ceph-mon[31227]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:58:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:19 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:58:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:19 smithi038 ceph-mon[31227]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:58:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:19 smithi038 ceph-mon[31227]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:58:19.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:19 smithi038 ceph-mon[31227]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:58:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:19 smithi179 ceph-mon[35781]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:58:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:19 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:58:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:19 smithi179 ceph-mon[35781]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:58:19.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:19 smithi179 ceph-mon[35781]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:58:19.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:19 smithi179 ceph-mon[35781]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:58:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:19 smithi047 ceph-mon[35327]: pgmap v865: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-19T05:58:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:19 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi038 to 177.0M 2024-09-19T05:58:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:19 smithi047 ceph-mon[35327]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-19T05:58:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:19 smithi047 ceph-mon[35327]: Unable to set osd_memory_target on smithi038 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T05:58:19.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:19 smithi047 ceph-mon[35327]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-19T05:58:20.070 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1118, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1120, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 294921, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 355588, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.091646882, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.030611353, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.024434422, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.027067282, 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:20.071 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1118, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.033134677, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000029637 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:20.072 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1037, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1272, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 256960, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 906615, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.117754481, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.048016145, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.025241408, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.034119920, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1271, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.045200454, 2024-09-19T05:58:20.073 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000035562 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 578752 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1436, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1432, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 905680, 2024-09-19T05:58:20.074 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 547394, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.175596996, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.080701027, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.051995880, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.033043553, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1431, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.082418222, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000057594 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.075 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 532384, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 194096 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:20.076 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5457, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.148216665, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000027160 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.077 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-09-19T05:58:20.078 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-09-19T05:58:20.079 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:20.080 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:20.081 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:20.082 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:20.083 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T05:58:20.084 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 5, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T05:58:20.085 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4195734, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 4195426, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 5 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 3, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 3, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-09-19T05:58:20.086 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 1, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 47224, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215328 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.087 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.088 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.090 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T05:58:20.092 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.093 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.094 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.096 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:20.097 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T05:58:20.098 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T05:58:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2084, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:20.101 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:20.102 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:20.103 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 190207, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 53, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 169, 2024-09-19T05:58:20.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 8512, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 13, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T05:58:20.105 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3260, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3260, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3260, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.420974072, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000435881 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T05:58:20.106 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3260, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3261 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3260, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 3260, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 2140, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 1118, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-09-19T05:58:20.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:20.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T05:58:20.109 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 91, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T05:58:20.110 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 5, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 1, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T05:58:20.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:20.112 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3587, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1114968, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3587, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3587, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1114968, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.113 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3260, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1435, 2024-09-19T05:58:20.114 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3260, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1435, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:20.115 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3260, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3260, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3260, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3260, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.116 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:20.117 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.120 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55a4fd8b0b40": { 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:20.121 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:20.122 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:21.073 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T05:58:21.073 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T05:58:21.073 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T05:58:21.073 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T05:58:21.335 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:21.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:21 smithi179 ceph-mon[35781]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2024-09-19T05:58:21.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:21 smithi047 ceph-mon[35327]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2024-09-19T05:58:21.885 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:21 smithi038 ceph-mon[31227]: pgmap v866: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1114, 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1116, 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 374249, 2024-09-19T05:58:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 354253, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.126045350, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.053234575, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.027669732, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.031715949, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1114, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.056164201, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000050416 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2483, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2477, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1084692, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 881183, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.195775917, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.065713781, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.066001589, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.056136379, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2477, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.089699317, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000036212 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 536736, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 197744 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 5, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 246, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1668, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 579973, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.019343958, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.014003090, 2024-09-19T05:58:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.002563222, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000065975, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 244, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.015594893, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000063913 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 579008 2024-09-19T05:58:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5461, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.134849447, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000024693 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T05:58:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T05:58:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-09-19T05:58:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-09-19T05:58:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T05:58:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 5, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T05:58:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4195734, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 4195426, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 5 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 3, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 3, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 1, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 0, 2024-09-19T05:58:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 47952, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215328 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-09-19T05:58:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T05:58:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T05:58:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2085, 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 198287, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 57, 2024-09-19T05:58:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 169, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 8512, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 13, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3266, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-09-19T05:58:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3266, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3266, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.594970883, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000488356 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3266, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3267 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3266, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 3266, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T05:58:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 2144, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 1120, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-09-19T05:58:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T05:58:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 91, 2024-09-19T05:58:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 6, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 8, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 2, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T05:58:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3598, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1116766, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3598, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3598, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1116766, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3266, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1435, 2024-09-19T05:58:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3266, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1435, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3266, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3266, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3266, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3266, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x560def040b40": { 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:22.042 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:22.791 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-19T05:58:22.791 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-19T05:58:22.791 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T05:58:22.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T05:58:23.051 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:23.586 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1444, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1438, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 909888, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 549758, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 5, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.155627115, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.061928479, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.053956844, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.031697873, 2024-09-19T05:58:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1438, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.063446658, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000044121 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 535920, 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 195824 2024-09-19T05:58:23.588 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1036, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1274, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 255336, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 907360, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.108346359, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.040637940, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.023494180, 2024-09-19T05:58:23.589 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.033724246, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1272, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.042140187, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000033129 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 578480 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.590 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1120, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 1121, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 294963, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 355843, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.091389124, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.031364412, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.024338652, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.026949346, 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1119, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.033330127, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000029785 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5410, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.135729488, 2024-09-19T05:58:23.593 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000025088 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.594 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-09-19T05:58:23.595 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 1, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-09-19T05:58:23.596 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-09-19T05:58:23.597 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:23.598 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 0 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-09-19T05:58:23.599 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:23.600 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-09-19T05:58:23.601 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 5, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 2, 2024-09-19T05:58:23.602 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 2, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 4195734, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 4195426, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 5 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.603 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-09-19T05:58:23.604 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 3, 2024-09-19T05:58:23.604 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 3, 2024-09-19T05:58:23.604 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-09-19T05:58:23.604 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 1, 2024-09-19T05:58:23.604 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 1, 2024-09-19T05:58:23.605 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-09-19T05:58:23.605 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 0, 2024-09-19T05:58:23.605 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 0, 2024-09-19T05:58:23.605 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-09-19T05:58:23.605 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 0, 2024-09-19T05:58:23.606 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 0, 2024-09-19T05:58:23.606 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 0, 2024-09-19T05:58:23.606 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 48284, 2024-09-19T05:58:23.606 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215328 2024-09-19T05:58:23.606 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.607 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-09-19T05:58:23.607 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 0, 2024-09-19T05:58:23.607 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:23.607 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 0, 2024-09-19T05:58:23.607 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 0, 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.608 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.609 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-09-19T05:58:23.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.609 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.610 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:23.610 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.610 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.610 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.611 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-09-19T05:58:23.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.612 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.612 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-09-19T05:58:23.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.614 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.615 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:23.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.615 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.616 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.618 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-09-19T05:58:23.618 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.619 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.619 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.619 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-09-19T05:58:23.620 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.620 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.620 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.620 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.620 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-09-19T05:58:23.621 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.622 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.622 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-09-19T05:58:23.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.623 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.623 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.624 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-09-19T05:58:23.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.624 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:23.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.626 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.626 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:23.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.628 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.628 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:23.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.629 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.629 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-09-19T05:58:23.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.630 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.630 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:23.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.631 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.632 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-09-19T05:58:23.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.632 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.633 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:23.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 2, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 1, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-09-19T05:58:23.635 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 2086, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:23.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:23.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 190167, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 52, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:23.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 169, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 8512, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 13, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T05:58:23.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 3266, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 21, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 3, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 3266, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3266, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 2.677951158, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000819948 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-09-19T05:58:23.641 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3266, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 3267 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 3266, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 3266, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 2144, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 1120, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 0, 2024-09-19T05:58:23.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-09-19T05:58:23.644 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 91, 2024-09-19T05:58:23.645 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 4, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 5, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 6, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 1, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 2, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 0 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-09-19T05:58:23.646 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 0, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.647 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3596, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 1116680, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3596, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3596, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1116680, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.648 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3266, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 1435, 2024-09-19T05:58:23.649 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3266, 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 1435, 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:23.650 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 3266, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 3266, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 3266, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 3266, 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:23.651 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:23.652 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55f1707e8b40": { 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:23.653 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.654 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:23.655 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:23.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:23 smithi179 ceph-mon[35781]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-09-19T05:58:23.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:23 smithi047 ceph-mon[35327]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-09-19T05:58:23.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:23 smithi038 ceph-mon[31227]: pgmap v867: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2024-09-19T05:58:24.426 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T05:58:24.426 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T05:58:24.426 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T05:58:24.426 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T05:58:24.695 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:25.276 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-19T05:58:25.276 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:25.276 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 322, 2024-09-19T05:58:25.276 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 319, 2024-09-19T05:58:25.276 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 509248, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 197040, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.047246693, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.024156393, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.024627785, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 318, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.023584882, 2024-09-19T05:58:25.277 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000074166 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 509248, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 197040 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 6, 2024-09-19T05:58:25.278 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 246, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 3312, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 200704, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.022643226, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.016056980, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.003582897, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:25.279 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 245, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.019837138, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000080967 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 200704 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.280 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.000298449, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000202493, 2024-09-19T05:58:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:25.282 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:25.283 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:25.284 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:25.285 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148704, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 29, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:25.286 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:25.287 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-19T05:58:25.288 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 324, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 463712, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 324, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 324, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 463712, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-19T05:58:25.289 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-19T05:58:25.290 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:25.290 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-19T05:58:25.290 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-19T05:58:25.723 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:25 smithi038 ceph-mon[31227]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:58:25.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:25 smithi047 ceph-mon[35327]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:58:25.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:25 smithi179 ceph-mon[35781]: pgmap v868: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-09-19T05:58:26.071 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T05:58:26.072 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T05:58:26.072 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T05:58:26.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T05:58:26.340 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:26.844 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 689, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 906, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 163613, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 182555, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 11, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.100533814, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.038975266, 2024-09-19T05:58:26.845 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.050457623, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.007511891, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 904, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.092436544, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000102252 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1525, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000474751, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000311 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:26.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 100, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 337, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 34143, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 664637, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 17, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 6, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.030071108, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.015155516, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.012691368, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000330810, 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:26.847 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 335, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.033158606, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000098980 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 163, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000055747, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000342 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 581360 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:26.848 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 457, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 443, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 595400, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 263242, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 11, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.061948102, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.025768157, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.042335971, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000533277, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 443, 2024-09-19T05:58:26.849 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.036786182, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000083038 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 156, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000039946, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000256 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 564016, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 200624 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-09-19T05:58:26.850 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 41, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001442166, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000035174 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:26.851 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000016930, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000008465 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 2, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 1, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000181558, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000181558 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.852 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 1, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 11, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 11, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 11, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 10, 2024-09-19T05:58:26.853 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 2, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 32, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 30, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 1, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-09-19T05:58:26.854 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 5, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 298, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:26.855 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:26.856 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 1 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 2, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-09-19T05:58:26.857 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_path": 14, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_inode": 28, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-09-19T05:58:26.858 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 4, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 4, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 2, 2024-09-19T05:58:26.859 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 2, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 2, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 4195426, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 0, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.006546373, 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.002182124 2024-09-19T05:58:26.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 0 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 14, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 14, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 0, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 12, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 13, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 1, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 11, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 11, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 0, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-09-19T05:58:26.861 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 1, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 1, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 50044, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 223520 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 3, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 2, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 4, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.862 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000181558, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000181558 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.863 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:26.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.866 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.867 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:26.869 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.871 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.872 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.873 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.874 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 0, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 1, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 1, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 0, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 1, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 1, 2024-09-19T05:58:26.875 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 77, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:26.876 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:26.877 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:26.878 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 263191, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 91, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 169, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:26.879 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 55080, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 213, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-09-19T05:58:26.880 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 38, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 11334, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 33, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 38, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 38, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.237994899, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.006263023 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 38, 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 60 2024-09-19T05:58:26.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 38, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 38, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 10, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 3, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 6, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-09-19T05:58:26.882 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 6, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 20, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-09-19T05:58:26.883 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-09-19T05:58:26.884 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 91, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 4, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 9, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 14, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 5, 2024-09-19T05:58:26.885 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 14, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 0, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 1 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 1, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 39 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-09-19T05:58:26.886 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 1242, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 659506, 2024-09-19T05:58:26.887 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 1242, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 1242, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 659506, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:26.888 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 38, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 11334, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 38, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 11334, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 38, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 38, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 38, 2024-09-19T05:58:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 38, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:26.891 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x55814c270960": { 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:26.892 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 4, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1122, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 4, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 3, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1122, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:26.894 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-19T05:58:27.679 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T05:58:27.680 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T05:58:27.680 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T05:58:27.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T05:58:27.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:27 smithi047 ceph-mon[35327]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:58:27.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:27 smithi038 ceph-mon[31227]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:58:27.933 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:27.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:27 smithi179 ceph-mon[35781]: pgmap v869: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-19T05:58:28.489 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-19T05:58:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 1450, 2024-09-19T05:58:28.489 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 1446, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 910000, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 553778, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.176473475, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.075869228, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.058791032, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.033780711, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1445, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.083991946, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000058125 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.490 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 534016, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 197936 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 8, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 248, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3868, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 583562, 2024-09-19T05:58:28.491 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 6, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.016751583, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.010315639, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.003785189, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000066096, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 247, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.018200106, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000073684 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.492 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 582864 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2162, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2164, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 551523, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 687523, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.145242480, 2024-09-19T05:58:28.493 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.031515498, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.042557460, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.062461897, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2162, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.057652518, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000026666 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:28.494 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5477, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.140289891, 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000025614 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-09-19T05:58:28.496 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-09-19T05:58:28.497 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-09-19T05:58:28.498 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 0, 2024-09-19T05:58:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:28.500 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:28.501 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:58:28.502 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-09-19T05:58:28.503 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 4, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 2, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 2, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-09-19T05:58:28.504 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 4195426, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 4195285, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 4 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 3, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 3, 2024-09-19T05:58:28.505 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 1, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 1, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 48656, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 215328 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 0, 2024-09-19T05:58:28.506 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.508 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-09-19T05:58:28.509 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.511 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-09-19T05:58:28.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.513 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.515 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.516 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.517 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 0, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 1, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 1, 2024-09-19T05:58:28.518 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 1, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 1304, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:28.519 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:28.521 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 182087, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 48, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-19T05:58:28.522 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 169, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 8512, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 13, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-19T05:58:28.523 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 3283, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 21, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 3, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 3283, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3283, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 1.737962524, 2024-09-19T05:58:28.524 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000529382 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3283, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 3284 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 3283, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 3283, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 0, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 2156, 2024-09-19T05:58:28.525 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 1125, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 1, 2024-09-19T05:58:28.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-09-19T05:58:28.527 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-09-19T05:58:28.528 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 91, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 4, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 4, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 6, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 2, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 0, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 2, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 0 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-09-19T05:58:28.529 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:28.530 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 3616, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1119876, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 3616, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:28.531 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 3616, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1119876, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:28.532 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 3283, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 1127, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 3283, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1127, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 3283, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 3283, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 3283, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 3283, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.534 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:28.535 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x56291d70ab40": { 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:28.536 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:28.537 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-19T05:58:29.289 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T05:58:29.289 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T05:58:29.289 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T05:58:29.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T05:58:29.539 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:29.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:29 smithi047 ceph-mon[35327]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-19T05:58:29.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:29 smithi179 ceph-mon[35781]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-19T05:58:29.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:29 smithi038 ceph-mon[31227]: pgmap v870: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 950, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 1214, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 686226, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 541982, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 10, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:30.014 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.140915833, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.061184460, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.075940528, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.007320219, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1213, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.075600466, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000062325 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1375, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000345078, 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000250 2024-09-19T05:58:30.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 546048, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 201936 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 38, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 278, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 10041, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 643292, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 19, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.035932643, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.014835505, 2024-09-19T05:58:30.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.017341092, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000243390, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 275, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.028852572, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000104918 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 40, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000007912, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000197 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 6960, 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 584112 2024-09-19T05:58:30.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 179, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 179, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 63669, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 91924, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 11, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.015956384, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.004832017, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.012198507, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000589555, 2024-09-19T05:58:30.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 179, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.028668132, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000160157 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 310, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000083975, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000270 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.019 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 49, 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001540998, 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000031448 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000012374, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000006187 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 1, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-09-19T05:58:30.021 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 1, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 11, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-09-19T05:58:30.022 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 10, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 10, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 10, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 28, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 28, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:30.023 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 2000, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 2, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 1, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 5, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 298, 2024-09-19T05:58:30.024 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:30.025 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 1, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:30.026 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_path": 14, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_inode": 14, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-09-19T05:58:30.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 5, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 5, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 2, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-09-19T05:58:30.028 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 2, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 2, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 4195734, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 0, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 4, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.007377251, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001844312 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 0 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.029 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 13, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 13, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 0, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 11, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 12, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 1, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 10, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 10, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 0, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 50048, 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 223520 2024-09-19T05:58:30.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 1, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 1, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 18, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.031 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-09-19T05:58:30.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-09-19T05:58:30.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.036 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.037 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.039 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:30.042 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 1, 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 2, 2024-09-19T05:58:30.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 1, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 1, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 1002, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:30.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:30.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:30.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 246903, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 82, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-19T05:58:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 169, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 50872, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 198, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:30.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 40, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 11732, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 32, 2024-09-19T05:58:30.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 40, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 40, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.219995277, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.005499881 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 40, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 62 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 40, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 0, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 40, 2024-09-19T05:58:30.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 10, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 4, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 7, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 6, 2024-09-19T05:58:30.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 20, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-09-19T05:58:30.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-09-19T05:58:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 91, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 4, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 9, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 14, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 5, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 14, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 0, 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 1 2024-09-19T05:58:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 1, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 26 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-09-19T05:58:30.055 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 1163, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 633010, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 1163, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:30.056 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 1163, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 633010, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:30.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 40, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 11732, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 40, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 11732, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:30.058 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 40, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 40, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 40, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 40, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.059 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-09-19T05:58:30.060 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x5625e42dc960": { 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 5, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1430, 2024-09-19T05:58:30.061 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 5, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 4, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1430, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:30.062 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-19T05:58:30.849 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T05:58:30.849 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T05:58:30.849 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T05:58:30.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T05:58:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 329, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 324, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 513056, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 199408, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.045365315, 2024-09-19T05:58:31.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.018725959, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.029417251, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 324, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.022939869, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000070802 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:31.587 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 513056, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 199408 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 246, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 200608, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.016442850, 2024-09-19T05:58:31.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.012023863, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001458547, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 244, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.014784255, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000060591 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:31.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 200608 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.000443848, 2024-09-19T05:58:31.590 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000355990, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:31.591 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:31.592 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:31.593 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:31.594 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 149735, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 30, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-19T05:58:31.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:31.596 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 327, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 464102, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 327, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-19T05:58:31.597 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 327, 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 464102, 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-19T05:58:31.598 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-19T05:58:31.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:31 smithi047 ceph-mon[35327]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:31.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:31 smithi038 ceph-mon[31227]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:31 smithi179 ceph-mon[35781]: pgmap v871: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:32.505 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-19T05:58:32.505 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-19T05:58:32.505 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T05:58:32.506 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T05:58:32.761 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:33.358 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 601, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 873, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 141697, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 347537, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 9, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.098186589, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.046197788, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.040921426, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.006702294, 2024-09-19T05:58:33.359 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 875, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.062242455, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000071134 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1402, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000372822, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000265 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 103, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 342, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 36833, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 673344, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 15, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.027861813, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.014291993, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.011250580, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000314593, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 341, 2024-09-19T05:58:33.361 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.035337237, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000103628 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 122, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000027325, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000223 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 582688 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 468, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 462, 2024-09-19T05:58:33.362 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 592091, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 262548, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.071768775, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.030552053, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.045722842, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000305722, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 457, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.030073850, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000065807 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 179, 2024-09-19T05:58:33.363 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000039007, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000217 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 561248, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 206192 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:33.364 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 44, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001439925, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000032725 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000012916, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000006458 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.365 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 1, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 1, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000109301, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000109301 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-09-19T05:58:33.366 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 10, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 10, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 10, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 1, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 1, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 1, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 28, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 28, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:33.367 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 5, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 298, 2024-09-19T05:58:33.368 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:33.369 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 1, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:33.370 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 14, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 14, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:33.371 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 5, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 2, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-09-19T05:58:33.372 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195734, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.008491455, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002122863 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-09-19T05:58:33.373 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 13, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 13, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 0, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 11, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 12, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 1, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 10, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 10, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 1, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 53192, 2024-09-19T05:58:33.374 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223520 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 1, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 1, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 18, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-09-19T05:58:33.375 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.376 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000109301, 2024-09-19T05:58:33.377 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000109301 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-09-19T05:58:33.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-09-19T05:58:33.380 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.381 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.382 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.383 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:33.384 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.385 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:33.386 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 1, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1, 2024-09-19T05:58:33.387 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1009, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:33.388 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:33.389 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:33.390 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 230759, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 75, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:33.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 51240, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 200, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-09-19T05:58:33.392 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 40, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 11732, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 28, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 40, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.197995834, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004949895 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40, 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 62 2024-09-19T05:58:33.393 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 40, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 40, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 4, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 7, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-09-19T05:58:33.394 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 6, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-09-19T05:58:33.395 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-09-19T05:58:33.396 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 91, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 8, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 4, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 14, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-09-19T05:58:33.397 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 1 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 26 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:33.398 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 1168, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 643143, 2024-09-19T05:58:33.399 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 1168, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1168, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 643143, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:33.400 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 40, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 11732, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 40, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 11732, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.401 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 40, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 40, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 40, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 40, 2024-09-19T05:58:33.402 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:33.403 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55f40bf66960": { 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 5, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1430, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 5, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1430, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.405 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:33.406 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-09-19T05:58:33.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:33 smithi038 ceph-mon[31227]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:33 smithi179 ceph-mon[35781]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:33.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:33 smithi047 ceph-mon[35327]: pgmap v872: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:34.199 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T05:58:34.199 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T05:58:34.199 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T05:58:34.200 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T05:58:34.445 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 53, 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 60, 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 13113, 2024-09-19T05:58:34.953 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 69206, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 18, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.022310366, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.005209933, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.014093398, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000634110, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 58, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.016553707, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000285408 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.954 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 52, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000009912, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000190 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 6032, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 7808 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 544, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 534, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 618589, 2024-09-19T05:58:34.955 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 293778, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 9, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.064312603, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.028608723, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.042547611, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000257491, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 534, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.048420077, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000090674 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:34.956 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 273, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000061273, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000224 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 556448, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 204464 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1188, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1339, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 293066, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 888962, 2024-09-19T05:58:34.957 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.132258254, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.066015271, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.063330654, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.006258809, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1337, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.070306833, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000052585 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2209, 2024-09-19T05:58:34.958 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000564638, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000255 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 494128 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 50, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001601717, 2024-09-19T05:58:34.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000032034 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000005574, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000002787 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 2, 2024-09-19T05:58:34.960 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 1, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000822586, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000822586 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 1, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-09-19T05:58:34.961 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 14, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 14, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 14, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 1, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 1, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 2, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 31, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 29, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 1, 2024-09-19T05:58:34.962 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 5, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 298, 2024-09-19T05:58:34.963 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 5, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 5, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:34.964 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 5, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 1 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 2, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-09-19T05:58:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 14, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 70, 2024-09-19T05:58:34.966 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 5, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-09-19T05:58:34.967 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 2, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195734, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-09-19T05:58:34.968 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.008110849, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002027712 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 17, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 17, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 0, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 15, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 16, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 1, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 14, 2024-09-19T05:58:34.969 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 14, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 1, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 53316, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223520 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 3, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 2, 2024-09-19T05:58:34.970 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 18, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000822586, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000822586 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:34.972 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.973 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.974 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.975 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.975 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.975 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-09-19T05:58:34.975 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.975 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.976 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.977 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.978 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:34.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:34.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 1, 2024-09-19T05:58:34.983 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1020, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:34.984 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:34.985 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:34.986 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 270951, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 88, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-09-19T05:58:34.987 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 68037, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 288, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-09-19T05:58:34.988 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 43, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 12198, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 43, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 43, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.208995609, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004860363 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 43, 2024-09-19T05:58:34.989 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 68 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 43, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 43, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 4, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 7, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-09-19T05:58:34.990 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 6, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-09-19T05:58:34.991 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-09-19T05:58:34.992 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 91, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 8, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-09-19T05:58:34.993 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 4, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 19, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 2 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-09-19T05:58:34.994 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 78 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:34.995 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 1781, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 741685, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 1781, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1781, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 741685, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 43, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 12198, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 43, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 12198, 2024-09-19T05:58:34.997 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:34.998 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 43, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 43, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 43, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 43, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:34.999 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:35.000 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x56460516a960": { 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 5, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1430, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 5, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:35.001 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4, 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1430, 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:35.002 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-09-19T05:58:35.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:35 smithi179 ceph-mon[35781]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:35.816 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T05:58:35.816 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T05:58:35.816 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T05:58:35.817 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T05:58:35.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:35 smithi038 ceph-mon[31227]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:35.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:35 smithi047 ceph-mon[35327]: pgmap v873: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-19T05:58:36.062 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:36.571 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-09-19T05:58:36.571 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 4596, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 4101, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1197744, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 8592061, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.387517603, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.153340878, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.162833110, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.074159757, 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:36.572 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4105, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.314790628, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000076684 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2659, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000571644, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000214 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:36.573 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2802, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 4426, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1117017, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 2204756, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 17, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 8, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.406547796, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.154978927, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.190288674, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.070425886, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4418, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.506372702, 2024-09-19T05:58:36.574 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000114615 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5218, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001110217, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000212 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1050304, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 2068032 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:36.575 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2382, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2176, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1291303, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1400276, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 72, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.246417710, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.108590690, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.167969452, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.003003399, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2176, 2024-09-19T05:58:36.576 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.210697313, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000096827 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3406, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000658830, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000193 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 575648, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 204720 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-09-19T05:58:36.577 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4974, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.365115709, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000274450 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:36.578 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 17, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000446773, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000026280 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 674, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 674, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 674, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.987917885, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004433112 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.579 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 12, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 25, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 25, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 24, 2024-09-19T05:58:36.580 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 2, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 2, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 2, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 733, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 724, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 35, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-09-19T05:58:36.581 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 4, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 234, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:36.582 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 2, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 646, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 668, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 599, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 752, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 707, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 2, 2024-09-19T05:58:36.583 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 628 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 6, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 1, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 3, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 2, 2024-09-19T05:58:36.584 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 14, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 194, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:36.585 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 2177, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 2177, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 4, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-09-19T05:58:36.586 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 4, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 11571713, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2163, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 4.186587588, 2024-09-19T05:58:36.587 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001935546 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 24, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 26, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 2, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 19, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 20, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 1, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 25, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 29, 2024-09-19T05:58:36.588 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 4, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 2, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 18, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 16, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 56364, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 231712 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 1892, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 674, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 24, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-09-19T05:58:36.589 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 18, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.010380631, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000576701 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 27, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.007468575, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000276613 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.590 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001031295, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000343765 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.591 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000200112, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000200112 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.592 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.593 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8, 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.014098947, 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001762368 2024-09-19T05:58:36.594 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000319350, 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000319350 2024-09-19T05:58:36.595 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.003758552, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001879276 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.002195500, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001097750 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.596 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.008645287, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004322643 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:36.597 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 601, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.914519921, 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004849450 2024-09-19T05:58:36.598 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.599 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.022939126, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002867390 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-09-19T05:58:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.002360589, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002360589 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-09-19T05:58:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 65, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 65, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1023, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:36.603 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-09-19T05:58:36.604 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 295196, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 100, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:36.605 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 99042, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 711, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-09-19T05:58:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3393, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 7397734, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 36, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3393, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3393, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 24.779477226, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.007303117 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-09-19T05:58:36.607 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3393, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3437 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3393, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 1, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 3392, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 11, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 2164, 2024-09-19T05:58:36.608 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 68, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 516, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 22, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:36.609 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 599, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-09-19T05:58:36.610 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 91, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 10, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 13, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-09-19T05:58:36.611 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 48, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 8 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 195 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.612 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 3, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 2, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 3, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 2, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 9776, 2024-09-19T05:58:36.613 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 2588728, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 9776, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 9776, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2588728, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:36.614 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3393, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 7397944, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3393, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 7397944, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.615 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3393, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3393, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3393, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3393, 2024-09-19T05:58:36.616 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 210, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:36.617 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 210, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55b9f01a6960": { 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:36.618 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2177, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 7377409, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2177, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2163, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 7377409, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:36.619 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-09-19T05:58:37.403 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T05:58:37.403 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T05:58:37.404 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T05:58:37.404 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T05:58:37.648 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:58:37.768 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:37 smithi179 ceph-mon[35781]: pgmap v874: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:58:37.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:37 smithi038 ceph-mon[31227]: pgmap v874: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:58:37.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:37 smithi047 ceph-mon[35327]: pgmap v874: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:58:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-09-19T05:58:38.124 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:58:38.124 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2186, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2188, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 4845225, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 695158, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.221231449, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.066636247, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.064821889, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.068555048, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2186, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.092768650, 2024-09-19T05:58:38.125 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042437 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 449, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 693, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 112962, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 726124, 2024-09-19T05:58:38.126 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.061991871, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.026889381, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.013463314, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.014951811, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 692, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.040349841, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000058309 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.127 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 585184 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1745, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1741, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 4072258, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 649363, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 3, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.177520147, 2024-09-19T05:58:38.128 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.073486638, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.060278309, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.037097986, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1740, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.072641509, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000041747 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.129 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 535808, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 201936 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6784, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.183952142, 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000027115 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.130 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.131 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 18, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 18, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-09-19T05:58:38.132 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 18, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 2, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-09-19T05:58:38.133 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 4, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 234, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:58:38.134 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-09-19T05:58:38.135 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-09-19T05:58:38.136 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-09-19T05:58:38.137 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 2177, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 4, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-09-19T05:58:38.138 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 11571713, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 11568880, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.139 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 2177 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 15, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 17, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 2, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 10, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 11, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 1, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 18, 2024-09-19T05:58:38.140 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 19, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 1, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 49020, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215328 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-09-19T05:58:38.141 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.142 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.143 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.144 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.145 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.146 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.147 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-09-19T05:58:38.148 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.149 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.150 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.151 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-09-19T05:58:38.152 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-09-19T05:58:38.153 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 2362, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:58:38.154 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-09-19T05:58:38.155 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-09-19T05:58:38.156 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 190972, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 54, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8960, 2024-09-19T05:58:38.157 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 59458, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 358, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-09-19T05:58:38.158 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 4036, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 26, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 4, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 4036, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4036, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.755962815, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000435075 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4036, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 4037 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.159 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 4036, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 4036, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 2256, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1777, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-09-19T05:58:38.160 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 2, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-09-19T05:58:38.161 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-09-19T05:58:38.162 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 91, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 5, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 6, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-09-19T05:58:38.163 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-09-19T05:58:38.164 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 4376, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 8616172, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 4376, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4376, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 8616172, 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:38.165 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 4036, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 7377419, 2024-09-19T05:58:38.166 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4036, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 7377419, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:38.167 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 4036, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 4036, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4036, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 4036, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:38.168 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.169 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x5628cb054b40": { 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-09-19T05:58:38.170 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-09-19T05:58:38.171 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-09-19T05:58:39.014 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T05:58:39.014 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T05:58:39.014 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T05:58:39.023 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T05:58:39.024 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:58:39.024 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:58:39.045 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-19T05:58:39.046 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:58:39.106 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:58:39.416 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:39.828 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:39 smithi038 ceph-mon[31227]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- 172.21.15.38:0/1349806029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904102790 msgr2=0x7f09041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 --2- 172.21.15.38:0/1349806029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904102790 0x7f09041067e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009900 tx=0x7f08f402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- 172.21.15.38:0/1349806029 shutdown_connections 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 --2- 172.21.15.38:0/1349806029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904106f10 0x7f090410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 --2- 172.21.15.38:0/1349806029 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904102790 0x7f09041067e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 --2- 172.21.15.38:0/1349806029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904101de0 0x7f09041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- 172.21.15.38:0/1349806029 >> 172.21.15.38:0/1349806029 conn(0x7f09040fd8c0 msgr2=0x7f09040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- 172.21.15.38:0/1349806029 shutdown_connections 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- 172.21.15.38:0/1349806029 wait complete. 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 Processor -- start 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.910+0000 7f090b611640 1 -- start start 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904102790 0x7f0904198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904106f10 0x7f090419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0904111f70 con 0x7f0904101de0 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0904111df0 con 0x7f0904106f10 2024-09-19T05:58:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f090b611640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f09041120f0 con 0x7f0904102790 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60198/0 (socket says 172.21.15.38:60198) 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 -- 172.21.15.38:0/2526573349 learned_addr learned my addr 172.21.15.38:0/2526573349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f0908bac640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904102790 0x7f0904198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f0909bae640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904106f10 0x7f090419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 -- 172.21.15.38:0/2526573349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904102790 msgr2=0x7f0904198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904102790 0x7f0904198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 -- 172.21.15.38:0/2526573349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904106f10 msgr2=0x7f090419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904106f10 0x7f090419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 -- 172.21.15.38:0/2526573349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090419d3e0 con 0x7f0904101de0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.911+0000 7f09093ad640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f090000d950 tx=0x7f090000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.912+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900017030 con 0x7f0904101de0 2024-09-19T05:58:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.912+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0900004020 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.912+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090419d6d0 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.912+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000047a0 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.912+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a4fb0 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.913+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09041021c0 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.914+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f09000041d0 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.914+0000 7f08fa7fc640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 0x7f08f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.915+0000 7f0908bac640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 0x7f08f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.915+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f09000cf640 con 0x7f0904101de0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.915+0000 7f0908bac640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 0x7f08f00a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009230 tx=0x7f08f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:39.919+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f090009bf40 con 0x7f0904101de0 2024-09-19T05:58:39.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:39 smithi179 ceph-mon[35781]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:39.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:39 smithi047 ceph-mon[35327]: pgmap v875: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.107+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0904199790 con 0x7f0904101de0 2024-09-19T05:58:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.108+0000 7f08fa7fc640 1 -- 172.21.15.38:0/2526573349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+592 (secure 0 0 0) 0x7f090009b840 con 0x7f0904101de0 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2329208823 2024-09-20T05:32:09.496697+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1466442042 2024-09-20T05:32:09.496697+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/3555821050 2024-09-20T05:32:09.496697+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1122120616 2024-09-20T05:31:45.752428+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2032678289 2024-09-20T05:31:45.752428+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2581276558 2024-09-20T05:31:45.752428+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2319061439 2024-09-20T05:32:09.496697+0000 2024-09-19T05:58:40.111 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/4202404879 2024-09-20T05:31:45.752428+0000 2024-09-19T05:58:40.112 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 msgr2=0x7f08f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 0x7f08f00a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009230 tx=0x7f08f40058d0 comp rx=0 tx=0).stop 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 msgr2=0x7f0904198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f090000d950 tx=0x7f090000de10 comp rx=0 tx=0).stop 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 shutdown_connections 2024-09-19T05:58:40.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f08f00a3240 0x7f08f00a5700 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0904106f10 0x7f090419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0904102790 0x7f0904198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 --2- 172.21.15.38:0/2526573349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0904101de0 0x7f0904198430 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 >> 172.21.15.38:0/2526573349 conn(0x7f09040fd8c0 msgr2=0x7f09040fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 shutdown_connections 2024-09-19T05:58:40.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:40.110+0000 7f090b611640 1 -- 172.21.15.38:0/2526573349 wait complete. 2024-09-19T05:58:40.826 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:40 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2526573349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:58:40.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T05:58:40.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:58:40.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T05:58:40.913 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T05:58:40.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:58:40.913 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T05:58:40.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:58:40.940 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-19T05:58:40.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:40 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2526573349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:58:40.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:40 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2526573349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:58:40.997 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T05:58:40.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:58:40.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:58:41.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:58:41.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-19T05:58:41.162 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T05:58:41.173 INFO:tasks.cephadm:Teardown begin 2024-09-19T05:58:41.173 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T05:58:41.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:58:41.433 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:41.848 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:41 smithi038 ceph-mon[31227]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:58:41.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:41 smithi179 ceph-mon[35781]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:58:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 -- 172.21.15.38:0/1459907775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc101420 msgr2=0x7f84cc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 --2- 172.21.15.38:0/1459907775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc101420 0x7f84cc10d950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009cd0 tx=0x7f84c402a8e0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 -- 172.21.15.38:0/1459907775 shutdown_connections 2024-09-19T05:58:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 --2- 172.21.15.38:0/1459907775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc101420 0x7f84cc10d950 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 --2- 172.21.15.38:0/1459907775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc100a80 0x7f84cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 --2- 172.21.15.38:0/1459907775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc106a80 0x7f84cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 -- 172.21.15.38:0/1459907775 >> 172.21.15.38:0/1459907775 conn(0x7f84cc0fc7a0 msgr2=0x7f84cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 -- 172.21.15.38:0/1459907775 shutdown_connections 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.929+0000 7f84d415a640 1 -- 172.21.15.38:0/1459907775 wait complete. 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 Processor -- start 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 -- start start 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc100a80 0x7f84cc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 0x7f84cc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc110e80 con 0x7f84cc106a80 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc110d00 con 0x7f84cc101420 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d415a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc111000 con 0x7f84cc100a80 2024-09-19T05:58:41.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d26f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d26f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60214/0 (socket says 172.21.15.38:60214) 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d16f5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 0x7f84cc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.930+0000 7f84d26f7640 1 -- 172.21.15.38:0/413816730 learned_addr learned my addr 172.21.15.38:0/413816730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d1ef6640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc100a80 0x7f84cc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 -- 172.21.15.38:0/413816730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc100a80 msgr2=0x7f84cc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc100a80 0x7f84cc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 -- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 msgr2=0x7f84cc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 -- 172.21.15.38:0/413816730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc19d3e0 con 0x7f84cc101420 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d26f7640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d16f5640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 0x7f84cc198970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00d6d0 tx=0x7f84bc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc017070 con 0x7f84cc101420 2024-09-19T05:58:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc19d6d0 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84bc009820 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004cd0 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.931+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc1a4fb0 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.932+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84cc100ee0 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.934+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f84bc0099d0 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.934+0000 7f84c2ffd640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 0x7f84b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.935+0000 7f84d1ef6640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 0x7f84b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.935+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f84bc0cfc40 con 0x7f84cc101420 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.935+0000 7f84d1ef6640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 0x7f84b80a5780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:41.938+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84bc09bef0 con 0x7f84cc101420 2024-09-19T05:58:41.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:41 smithi047 ceph-mon[35327]: pgmap v876: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-19T05:58:42.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.149+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84cc10a030 con 0x7f84cc101420 2024-09-19T05:58:42.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.150+0000 7f84c2ffd640 1 -- 172.21.15.38:0/413816730 <== mon.2 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f84bc0a01f0 con 0x7f84cc101420 2024-09-19T05:58:42.153 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:42.153 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:58:42.153 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:58:42.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 msgr2=0x7f84b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:42.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 0x7f84b80a5780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c800b390 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 msgr2=0x7f84cc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 0x7f84cc198970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00d6d0 tx=0x7f84bc00db90 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 shutdown_connections 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f84b80a32c0 0x7f84b80a5780 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84cc106a80 0x7f84cc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84cc101420 0x7f84cc198970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 --2- 172.21.15.38:0/413816730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84cc100a80 0x7f84cc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 >> 172.21.15.38:0/413816730 conn(0x7f84cc0fc7a0 msgr2=0x7f84cc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 shutdown_connections 2024-09-19T05:58:42.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:42.152+0000 7f84d415a640 1 -- 172.21.15.38:0/413816730 wait complete. 2024-09-19T05:58:42.871 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:42 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/413816730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:42.904 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:58:42.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:42 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/413816730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:42.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:42 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/413816730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:43.164 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:43.584 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:43 smithi038 ceph-mon[31227]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:43.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 -- 172.21.15.38:0/2669112237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 msgr2=0x7f3a0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 --2- 172.21.15.38:0/2669112237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c103110 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009930 tx=0x7f3a0802a420 comp rx=0 tx=0).stop 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 -- 172.21.15.38:0/2669112237 shutdown_connections 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 --2- 172.21.15.38:0/2669112237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 0x7f3a0c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 --2- 172.21.15.38:0/2669112237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 0x7f3a0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 --2- 172.21.15.38:0/2669112237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c103110 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 -- 172.21.15.38:0/2669112237 >> 172.21.15.38:0/2669112237 conn(0x7f3a0c0fc7d0 msgr2=0x7f3a0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 -- 172.21.15.38:0/2669112237 shutdown_connections 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.666+0000 7f3a142b3640 1 -- 172.21.15.38:0/2669112237 wait complete. 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 Processor -- start 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 -- start start 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:43.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 0x7f3a0c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 0x7f3a0c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c1120a0 con 0x7f3a0c102d30 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c111f20 con 0x7f3a0c1082a0 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a142b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c112220 con 0x7f3a0c1036e0 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a1204f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a1204f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60220/0 (socket says 172.21.15.38:60220) 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.667+0000 7f3a1204f640 1 -- 172.21.15.38:0/3272450758 learned_addr learned my addr 172.21.15.38:0/3272450758 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1184e640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 0x7f3a0c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a12850640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 0x7f3a0c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 -- 172.21.15.38:0/3272450758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 msgr2=0x7f3a0c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 0x7f3a0c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 -- 172.21.15.38:0/3272450758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 msgr2=0x7f3a0c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 0x7f3a0c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 -- 172.21.15.38:0/3272450758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c19d550 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a1204f640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009230 tx=0x7f3a08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0803f030 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c19d7e0 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a08003ad0 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08035070 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.668+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c1008c0 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.670+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0c06a850 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.670+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3a08042030 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.671+0000 7f3a037fe640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 0x7f39f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.671+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f3a080f0310 con 0x7f3a0c102d30 2024-09-19T05:58:43.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.671+0000 7f3a1184e640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 0x7f39f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.672+0000 7f3a1184e640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 0x7f39f80a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0fee30 tx=0x7f39fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.675+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a080bcc70 con 0x7f3a0c102d30 2024-09-19T05:58:43.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.892+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a0c199900 con 0x7f3a0c102d30 2024-09-19T05:58:43.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.892+0000 7f3a037fe640 1 -- 172.21.15.38:0/3272450758 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f3a080dc020 con 0x7f3a0c102d30 2024-09-19T05:58:43.895 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:43.895 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"btime":"2024-09-19T05:41:31:277929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/418551929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":418551929},{"type":"v1","addr":"172.21.15.47:6833","nonce":418551929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1823768256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1823768256},{"type":"v1","addr":"172.21.15.38:6835","nonce":1823768256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:39:02.544314+0000","modified":"2024-09-19T05:41:30.491887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24496,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14547":{"gid":14547,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6837/4064878755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":4064878755},{"type":"v1","addr":"172.21.15.38:6837","nonce":4064878755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14598":{"gid":14598,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6839/2963619728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":2963619728},{"type":"v1","addr":"172.21.15.38:6839","nonce":2963619728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6841/3698066275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":3698066275},{"type":"v1","addr":"172.21.15.38:6841","nonce":3698066275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6835/3758113926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3758113926},{"type":"v1","addr":"172.21.15.47:6835","nonce":3758113926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2514682242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2514682242},{"type":"v1","addr":"172.21.15.179:6833","nonce":2514682242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.47:6837/2962930600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2962930600},{"type":"v1","addr":"172.21.15.47:6837","nonce":2962930600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":19,"addr":"172.21.15.179:6835/994628887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":994628887},{"type":"v1","addr":"172.21.15.179:6835","nonce":994628887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.179:6837/164318179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":164318179},{"type":"v1","addr":"172.21.15.179:6837","nonce":164318179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.47:6839/1610791508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1610791508},{"type":"v1","addr":"172.21.15.47:6839","nonce":1610791508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.179:6839/2081935059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2081935059},{"type":"v1","addr":"172.21.15.179:6839","nonce":2081935059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24455,24481,24475,24485,24496]},"id":1}]} 2024-09-19T05:58:43.896 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 msgr2=0x7f39f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 0x7f39f80a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0fee30 tx=0x7f39fc00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 msgr2=0x7f3a0c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009230 tx=0x7f3a08004270 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 shutdown_connections 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f39f80a3240 0x7f39f80a5700 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1082a0 0x7f3a0c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0c1036e0 0x7f3a0c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 --2- 172.21.15.38:0/3272450758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a0c102d30 0x7f3a0c1985a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 >> 172.21.15.38:0/3272450758 conn(0x7f3a0c0fc7d0 msgr2=0x7f3a0c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:43.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 shutdown_connections 2024-09-19T05:58:43.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:43.896+0000 7f3a142b3640 1 -- 172.21.15.38:0/3272450758 wait complete. 2024-09-19T05:58:43.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:43 smithi179 ceph-mon[35781]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:43.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:43 smithi047 ceph-mon[35327]: pgmap v877: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T05:58:44.728 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:44 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3272450758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:44.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:58:44.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:44 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3272450758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:44.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:44 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3272450758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:58:45.061 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:45.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 -- 172.21.15.38:0/1323224940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 msgr2=0x7feba81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:45.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 --2- 172.21.15.38:0/1323224940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81067e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7feb98009900 tx=0x7feb9802a410 comp rx=0 tx=0).stop 2024-09-19T05:58:45.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 -- 172.21.15.38:0/1323224940 shutdown_connections 2024-09-19T05:58:45.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 --2- 172.21.15.38:0/1323224940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 0x7feba810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 --2- 172.21.15.38:0/1323224940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81067e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 --2- 172.21.15.38:0/1323224940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 0x7feba81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 -- 172.21.15.38:0/1323224940 >> 172.21.15.38:0/1323224940 conn(0x7feba80fd8c0 msgr2=0x7feba80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 -- 172.21.15.38:0/1323224940 shutdown_connections 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.593+0000 7febafb1b640 1 -- 172.21.15.38:0/1323224940 wait complete. 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 Processor -- start 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 -- start start 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 0x7feba81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 0x7feba819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feba8111f70 con 0x7feba8102790 2024-09-19T05:58:45.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feba8111df0 con 0x7feba8106f10 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febafb1b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feba81120f0 con 0x7feba8101de0 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60242/0 (socket says 172.21.15.38:60242) 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 -- 172.21.15.38:0/2941189726 learned_addr learned my addr 172.21.15.38:0/2941189726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febae0b8640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 0x7feba819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad8b7640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 0x7feba81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 -- 172.21.15.38:0/2941189726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 msgr2=0x7feba81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 0x7feba81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.594+0000 7febad0b6640 1 -- 172.21.15.38:0/2941189726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 msgr2=0x7feba819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7febad0b6640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 0x7feba819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7febad0b6640 1 -- 172.21.15.38:0/2941189726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba819d460 con 0x7feba8102790 2024-09-19T05:58:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7febad0b6640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9803f030 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98003ce0 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba819d6f0 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98035070 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.595+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81a5030 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.596+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba81021c0 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.597+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7feb98042030 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.598+0000 7feb9effd640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 0x7feb940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.598+0000 7febad8b7640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 0x7feb940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.598+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7feb980f0490 con 0x7feba8102790 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.598+0000 7febad8b7640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 0x7feb940a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feba4005fd0 tx=0x7feba4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:45.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.601+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb980bceb0 con 0x7feba8102790 2024-09-19T05:58:45.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.779+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feba810a150 con 0x7feba8102790 2024-09-19T05:58:45.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:45 smithi038 ceph-mon[31227]: pgmap v878: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:58:45.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.781+0000 7feb9effd640 1 -- 172.21.15.38:0/2941189726 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24875 (secure 0 0 0) 0x7feb980bc7b0 con 0x7feba8102790 2024-09-19T05:58:45.783 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-19T05:58:45.783 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"fsid":"44ff0e58-7648-11ef-bceb-c7b262605968","created":"2024-09-19T05:31:23.688213+0000","modified":"2024-09-19T05:41:11.190669+0000","last_up_change":"2024-09-19T05:38:10.851186+0000","last_in_change":"2024-09-19T05:37:46.925194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:34:23.998201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:38:58.787912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:39:00.870380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T05:39:07.367942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6d08454b-46b5-4ad6-8e7f-3c79c155e73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6803","nonce":3037922100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6805","nonce":3037922100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6809","nonce":3037922100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":3037922100},{"type":"v1","addr":"172.21.15.38:6807","nonce":3037922100}]},"public_addr":"172.21.15.38:6803/3037922100","cluster_addr":"172.21.15.38:6805/3037922100","heartbeat_back_addr":"172.21.15.38:6809/3037922100","heartbeat_front_addr":"172.21.15.38:6807/3037922100","state":["exists","up"]},{"osd":1,"uuid":"7156f9e6-19c1-4422-89f5-97209c281838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6801","nonce":716320607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6803","nonce":716320607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6807","nonce":716320607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":716320607},{"type":"v1","addr":"172.21.15.47:6805","nonce":716320607}]},"public_addr":"172.21.15.47:6801/716320607","cluster_addr":"172.21.15.47:6803/716320607","heartbeat_back_addr":"172.21.15.47:6807/716320607","heartbeat_front_addr":"172.21.15.47:6805/716320607","state":["exists","up"]},{"osd":2,"uuid":"94fbc241-9df4-4189-9369-72b359a64f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6801","nonce":4020882332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6803","nonce":4020882332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6807","nonce":4020882332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4020882332},{"type":"v1","addr":"172.21.15.179:6805","nonce":4020882332}]},"public_addr":"172.21.15.179:6801/4020882332","cluster_addr":"172.21.15.179:6803/4020882332","heartbeat_back_addr":"172.21.15.179:6807/4020882332","heartbeat_front_addr":"172.21.15.179:6805/4020882332","state":["exists","up"]},{"osd":3,"uuid":"8c145d0a-468b-40cd-bc68-2214559e2fdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6811","nonce":1962488545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6813","nonce":1962488545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6817","nonce":1962488545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1962488545},{"type":"v1","addr":"172.21.15.38:6815","nonce":1962488545}]},"public_addr":"172.21.15.38:6811/1962488545","cluster_addr":"172.21.15.38:6813/1962488545","heartbeat_back_addr":"172.21.15.38:6817/1962488545","heartbeat_front_addr":"172.21.15.38:6815/1962488545","state":["exists","up"]},{"osd":4,"uuid":"2ea25802-a31e-4e01-bccf-183ee703afe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6809","nonce":3118152095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6811","nonce":3118152095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6815","nonce":3118152095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":3118152095},{"type":"v1","addr":"172.21.15.47:6813","nonce":3118152095}]},"public_addr":"172.21.15.47:6809/3118152095","cluster_addr":"172.21.15.47:6811/3118152095","heartbeat_back_addr":"172.21.15.47:6815/3118152095","heartbeat_front_addr":"172.21.15.47:6813/3118152095","state":["exists","up"]},{"osd":5,"uuid":"03dfe06a-5da0-4925-a09a-9fea434b0633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6809","nonce":322546150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6811","nonce":322546150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6815","nonce":322546150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":322546150},{"type":"v1","addr":"172.21.15.179:6813","nonce":322546150}]},"public_addr":"172.21.15.179:6809/322546150","cluster_addr":"172.21.15.179:6811/322546150","heartbeat_back_addr":"172.21.15.179:6815/322546150","heartbeat_front_addr":"172.21.15.179:6813/322546150","state":["exists","up"]},{"osd":6,"uuid":"e40f682d-ec98-470b-b2e2-19946849a6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6819","nonce":2208960184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6821","nonce":2208960184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6825","nonce":2208960184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":2208960184},{"type":"v1","addr":"172.21.15.38:6823","nonce":2208960184}]},"public_addr":"172.21.15.38:6819/2208960184","cluster_addr":"172.21.15.38:6821/2208960184","heartbeat_back_addr":"172.21.15.38:6825/2208960184","heartbeat_front_addr":"172.21.15.38:6823/2208960184","state":["exists","up"]},{"osd":7,"uuid":"a99cd83f-58cb-47b9-bcea-35e82ceacbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6817","nonce":2705740748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6819","nonce":2705740748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6823","nonce":2705740748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2705740748},{"type":"v1","addr":"172.21.15.47:6821","nonce":2705740748}]},"public_addr":"172.21.15.47:6817/2705740748","cluster_addr":"172.21.15.47:6819/2705740748","heartbeat_back_addr":"172.21.15.47:6823/2705740748","heartbeat_front_addr":"172.21.15.47:6821/2705740748","state":["exists","up"]},{"osd":8,"uuid":"1306f615-40c3-4be2-b882-b665b1b01588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6817","nonce":2844755056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6819","nonce":2844755056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6823","nonce":2844755056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2844755056},{"type":"v1","addr":"172.21.15.179:6821","nonce":2844755056}]},"public_addr":"172.21.15.179:6817/2844755056","cluster_addr":"172.21.15.179:6819/2844755056","heartbeat_back_addr":"172.21.15.179:6823/2844755056","heartbeat_front_addr":"172.21.15.179:6821/2844755056","state":["exists","up"]},{"osd":9,"uuid":"f3874c68-e48a-452f-a6f4-c349235e29da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6827","nonce":3187320508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6829","nonce":3187320508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6833","nonce":3187320508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3187320508},{"type":"v1","addr":"172.21.15.38:6831","nonce":3187320508}]},"public_addr":"172.21.15.38:6827/3187320508","cluster_addr":"172.21.15.38:6829/3187320508","heartbeat_back_addr":"172.21.15.38:6833/3187320508","heartbeat_front_addr":"172.21.15.38:6831/3187320508","state":["exists","up"]},{"osd":10,"uuid":"1a1ddce0-f377-454c-a5f2-b8ecad532909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6825","nonce":1633858254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6827","nonce":1633858254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6831","nonce":1633858254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1633858254},{"type":"v1","addr":"172.21.15.47:6829","nonce":1633858254}]},"public_addr":"172.21.15.47:6825/1633858254","cluster_addr":"172.21.15.47:6827/1633858254","heartbeat_back_addr":"172.21.15.47:6831/1633858254","heartbeat_front_addr":"172.21.15.47:6829/1633858254","state":["exists","up"]},{"osd":11,"uuid":"7cda5e25-4843-426b-a917-89f6546f30a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6825","nonce":1092638646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6827","nonce":1092638646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6831","nonce":1092638646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1092638646},{"type":"v1","addr":"172.21.15.179:6829","nonce":1092638646}]},"public_addr":"172.21.15.179:6825/1092638646","cluster_addr":"172.21.15.179:6827/1092638646","heartbeat_back_addr":"172.21.15.179:6831/1092638646","heartbeat_front_addr":"172.21.15.179:6829/1092638646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:31.611510+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:33:55.299327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:20.482865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:34:45.380369+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:11.079777+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:35:36.232707+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:02.531400+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:27.705820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:36:52.359342+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:18.499887+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:37:43.684335+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:38:08.984289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/2329208823":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1466442042":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/3555821050":"2024-09-20T05:32:09.496697+0000","172.21.15.38:0/1122120616":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2032678289":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2581276558":"2024-09-20T05:31:45.752428+0000","172.21.15.38:6801/4202404879":"2024-09-20T05:31:45.752428+0000","172.21.15.38:0/2319061439":"2024-09-20T05:32:09.496697+0000","172.21.15.38:6800/4202404879":"2024-09-20T05:31:45.752428+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 msgr2=0x7feb940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 0x7feb940a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feba4005fd0 tx=0x7feba4005850 comp rx=0 tx=0).stop 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 msgr2=0x7feba81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb980042f0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 shutdown_connections 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feb940a3240 0x7feb940a5700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feba8106f10 0x7feba819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feba8102790 0x7feba81989f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 --2- 172.21.15.38:0/2941189726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feba8101de0 0x7feba81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:45.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 >> 172.21.15.38:0/2941189726 conn(0x7feba80fd8c0 msgr2=0x7feba80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:45.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 shutdown_connections 2024-09-19T05:58:45.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:45.783+0000 7febafb1b640 1 -- 172.21.15.38:0/2941189726 wait complete. 2024-09-19T05:58:45.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:45 smithi179 ceph-mon[35781]: pgmap v878: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:58:45.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:45 smithi047 ceph-mon[35327]: pgmap v878: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T05:58:46.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T05:58:46.740 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:46.861 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:46 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2941189726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:46.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:46 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2941189726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:46.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:46 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2941189726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 -- 172.21.15.38:0/2144938222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650101de0 msgr2=0x7f76501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 --2- 172.21.15.38:0/2144938222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650101de0 0x7f76501021c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7640009900 tx=0x7f764002a410 comp rx=0 tx=0).stop 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 -- 172.21.15.38:0/2144938222 shutdown_connections 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 --2- 172.21.15.38:0/2144938222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 0x7f765010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 --2- 172.21.15.38:0/2144938222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650102790 0x7f76501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 --2- 172.21.15.38:0/2144938222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650101de0 0x7f76501021c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 -- 172.21.15.38:0/2144938222 >> 172.21.15.38:0/2144938222 conn(0x7f76500fd8c0 msgr2=0x7f76500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 -- 172.21.15.38:0/2144938222 shutdown_connections 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.239+0000 7f765671a640 1 -- 172.21.15.38:0/2144938222 wait complete. 2024-09-19T05:58:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 Processor -- start 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 -- start start 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 0x7f7650198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 0x7f765019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7650111f70 con 0x7f7650102790 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7650111df0 con 0x7f7650101de0 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f765671a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f76501120f0 con 0x7f7650106f10 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f7647fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 0x7f7650198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f7647fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60264/0 (socket says 172.21.15.38:60264) 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f7647fff640 1 -- 172.21.15.38:0/4270640338 learned_addr learned my addr 172.21.15.38:0/4270640338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f7654cb7640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 0x7f765019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 -- 172.21.15.38:0/4270640338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 msgr2=0x7f765019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 0x7f765019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 -- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 msgr2=0x7f76501989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f764ffff640 1 -- 172.21.15.38:0/4270640338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765019d420 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.240+0000 7f7647fff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f764ffff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 0x7f7650198470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7640005940 tx=0x7f7640004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764003f030 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765019d6b0 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7640003ad0 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640035070 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.241+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76501a4ff0 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.242+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76501032c0 con 0x7f7650101de0 2024-09-19T05:58:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.243+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7640042030 con 0x7f7650101de0 2024-09-19T05:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.244+0000 7f764dffb640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 0x7f763c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.244+0000 7f7647fff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 0x7f763c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.244+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9187+0+0 (secure 0 0 0) 0x7f76400f0520 con 0x7f7650101de0 2024-09-19T05:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.245+0000 7f7647fff640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 0x7f763c0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76500fdce0 tx=0x7f7638005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:47.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.248+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f76400bc7d0 con 0x7f7650101de0 2024-09-19T05:58:47.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.458+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f765010a0f0 con 0x7f7650101de0 2024-09-19T05:58:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.467+0000 7f764dffb640 1 -- 172.21.15.38:0/4270640338 <== mon.2 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f76400c0ad0 con 0x7f7650101de0 2024-09-19T05:58:47.469 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T05:58:47.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.470+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 msgr2=0x7f763c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:47.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.470+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 0x7f763c0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76500fdce0 tx=0x7f7638005f90 comp rx=0 tx=0).stop 2024-09-19T05:58:47.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.470+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 msgr2=0x7f7650198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:47.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.470+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 0x7f7650198470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7640005940 tx=0x7f7640004270 comp rx=0 tx=0).stop 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 shutdown_connections 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f763c0a3240 0x7f763c0a5700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7650106f10 0x7f765019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7650102790 0x7f76501989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 --2- 172.21.15.38:0/4270640338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7650101de0 0x7f7650198470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 >> 172.21.15.38:0/4270640338 conn(0x7f76500fd8c0 msgr2=0x7f76500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 shutdown_connections 2024-09-19T05:58:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:47.471+0000 7f765671a640 1 -- 172.21.15.38:0/4270640338 wait complete. 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a[31223]: 2024-09-19T05:58:47.460+0000 7fd2d087f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: pgmap v879: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/4270640338' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:58:47.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mon[31227]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mds[69186]: -- 172.21.15.38:0/1239280576 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581ea235c20 con 0x5581ea213400 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: starting mds.d at 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: starting mds.g at 2024-09-19T05:58:47.895 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.896 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.896 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-mds[70371]: -- 172.21.15.38:0/4058870566 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565505dc9860 con 0x565505da6800 2024-09-19T05:58:47.896 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 05:58:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: starting mds.a at 2024-09-19T05:58:47.947 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.947 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.947 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mds[66316]: -- 172.21.15.179:0/479219888 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda0b2f860 con 0x55dda0b0d400 2024-09-19T05:58:47.947 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: starting mds.i at 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: pgmap v879: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/4270640338' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:58:47.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mon[35781]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mds[65712]: -- 172.21.15.179:0/605471426 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c574d1860 con 0x556c574af400 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: starting mds.f at 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c[66008]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c[66008]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mds[66012]: -- 172.21.15.179:0/1540865766 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634f40b9860 con 0x5634f4096800 2024-09-19T05:58:47.949 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c[66008]: starting mds.c at 2024-09-19T05:58:47.950 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l[65406]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.950 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l[65406]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.950 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-mds[65410]: -- 172.21.15.179:0/1759912451 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bcd7c07860 con 0x55bcd7be4800 2024-09-19T05:58:47.950 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 05:58:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l[65406]: starting mds.l at 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: starting mds.h at 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.976 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mds[64582]: -- 172.21.15.47:0/3777533371 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a17debdc20 con 0x55a17de9a800 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: starting mds.e at 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: pgmap v879: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/4270640338' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:58:47.977 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mon[35327]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:58:47.978 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b[65195]: ignoring --setuser ceph since I am not root 2024-09-19T05:58:47.978 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b[65195]: ignoring --setgroup ceph since I am not root 2024-09-19T05:58:47.978 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-mds[65199]: -- 172.21.15.47:0/1705908129 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563059cd9860 con 0x563059cb6800 2024-09-19T05:58:47.978 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 05:58:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b[65195]: starting mds.b at 2024-09-19T05:58:48.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T05:58:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 -- 172.21.15.38:0/3669082330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488102790 msgr2=0x7f94881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 --2- 172.21.15.38:0/3669082330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488102790 0x7f94881067e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9478009900 tx=0x7f947802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 -- 172.21.15.38:0/3669082330 shutdown_connections 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 --2- 172.21.15.38:0/3669082330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488106f10 0x7f948810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 --2- 172.21.15.38:0/3669082330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488102790 0x7f94881067e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 --2- 172.21.15.38:0/3669082330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488101de0 0x7f94881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 -- 172.21.15.38:0/3669082330 >> 172.21.15.38:0/3669082330 conn(0x7f94880fd8c0 msgr2=0x7f94880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 -- 172.21.15.38:0/3669082330 shutdown_connections 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.998+0000 7f948f422640 1 -- 172.21.15.38:0/3669082330 wait complete. 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 Processor -- start 2024-09-19T05:58:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 -- start start 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488101de0 0x7f94881983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488102790 0x7f9488198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9488111f70 con 0x7f9488106f10 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9488111df0 con 0x7f9488102790 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948f422640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94881120f0 con 0x7f9488101de0 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54586/0 (socket says 172.21.15.38:54586) 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 -- 172.21.15.38:0/1687220777 learned_addr learned my addr 172.21.15.38:0/1687220777 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d1be640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488101de0 0x7f94881983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948c9bd640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488102790 0x7f9488198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 -- 172.21.15.38:0/1687220777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488101de0 msgr2=0x7f94881983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488101de0 0x7f94881983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 -- 172.21.15.38:0/1687220777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488102790 msgr2=0x7f9488198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488102790 0x7f9488198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:48.999+0000 7f948d9bf640 1 -- 172.21.15.38:0/1687220777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948819d380 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f948d9bf640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f948000cce0 tx=0x7f948000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948000f450 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948000fad0 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948819d670 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480018b60 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.000+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a4f50 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.001+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94881021c0 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.002+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9480018d00 con 0x7f9488106f10 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.003+0000 7f947e7fc640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 0x7f94740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.003+0000 7f948d1be640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 0x7f94740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.003+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9617+0+0 (secure 0 0 0) 0x7f94800d03e0 con 0x7f9488106f10 2024-09-19T05:58:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.003+0000 7f948d1be640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 0x7f94740a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f948400abe0 tx=0x7f948400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.007+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9480014030 con 0x7f9488106f10 2024-09-19T05:58:49.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.218+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f94881a51b0 con 0x7f9488106f10 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: pgmap v881: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6834/3977359650,v1:172.21.15.47:6835/3977359650] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6832/619526313,v1:172.21.15.179:6834/619526313] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6833/2409838115,v1:172.21.15.179:6835/2409838115] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6836/331467361,v1:172.21.15.47:6837/331467361] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.47:6838/2488791931,v1:172.21.15.47:6839/2488791931] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6837/3746148561,v1:172.21.15.38:6839/3746148561] up:boot 2024-09-19T05:58:49.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6836/1693392542,v1:172.21.15.38:6838/1693392542] up:boot 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6836/36190265,v1:172.21.15.179:6837/36190265] up:boot 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.179:6838/2783210168,v1:172.21.15.179:6839/2783210168] up:boot 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: mds.? [v2:172.21.15.38:6840/3132552306,v1:172.21.15.38:6841/3132552306] up:boot 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:58:49.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:58:49.477 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:49 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:58:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.548+0000 7f947e7fc640 1 -- 172.21.15.38:0/1687220777 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f948009cb30 con 0x7f9488106f10 2024-09-19T05:58:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.550+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 msgr2=0x7f94740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:49.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 0x7f94740a5780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f948400abe0 tx=0x7f948400a2d0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 msgr2=0x7f948819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f948000cce0 tx=0x7f948000a620 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 shutdown_connections 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f94740a32c0 0x7f94740a5780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9488106f10 0x7f948819cca0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9488102790 0x7f9488198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 --2- 172.21.15.38:0/1687220777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9488101de0 0x7f94881983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 >> 172.21.15.38:0/1687220777 conn(0x7f94880fd8c0 msgr2=0x7f94880ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 shutdown_connections 2024-09-19T05:58:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:49.551+0000 7f948f422640 1 -- 172.21.15.38:0/1687220777 wait complete. 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: pgmap v881: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6834/3977359650,v1:172.21.15.47:6835/3977359650] up:boot 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6832/619526313,v1:172.21.15.179:6834/619526313] up:boot 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6833/2409838115,v1:172.21.15.179:6835/2409838115] up:boot 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6836/331467361,v1:172.21.15.47:6837/331467361] up:boot 2024-09-19T05:58:49.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.47:6838/2488791931,v1:172.21.15.47:6839/2488791931] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6837/3746148561,v1:172.21.15.38:6839/3746148561] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6836/1693392542,v1:172.21.15.38:6838/1693392542] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6836/36190265,v1:172.21.15.179:6837/36190265] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.179:6838/2783210168,v1:172.21.15.179:6839/2783210168] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: mds.? [v2:172.21.15.38:6840/3132552306,v1:172.21.15.38:6841/3132552306] up:boot 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:58:49.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:58:49.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:58:49.896 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:49 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:58:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: pgmap v881: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-19T05:58:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6834/3977359650,v1:172.21.15.47:6835/3977359650] up:boot 2024-09-19T05:58:49.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6832/619526313,v1:172.21.15.179:6834/619526313] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6833/2409838115,v1:172.21.15.179:6835/2409838115] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6836/331467361,v1:172.21.15.47:6837/331467361] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.47:6838/2488791931,v1:172.21.15.47:6839/2488791931] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6837/3746148561,v1:172.21.15.38:6839/3746148561] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6836/1693392542,v1:172.21.15.38:6838/1693392542] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6836/36190265,v1:172.21.15.179:6837/36190265] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.179:6838/2783210168,v1:172.21.15.179:6839/2783210168] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: mds.? [v2:172.21.15.38:6840/3132552306,v1:172.21.15.38:6841/3132552306] up:boot 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:58:49.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='mgr.14150 172.21.15.38:0/608689375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:58:49.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:49 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:58:50.425 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T05:58:50.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:50 smithi038 ceph-mon[31227]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:58:50.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:50 smithi038 ceph-mon[31227]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:58:50.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:50 smithi038 ceph-mon[31227]: Cluster is now healthy 2024-09-19T05:58:50.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:50 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:58:50.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:50 smithi038 ceph-mon[31227]: fsmap 12 up:standby 2024-09-19T05:58:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:50.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:50 smithi047 ceph-mon[35327]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:58:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:50 smithi047 ceph-mon[35327]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:58:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:50 smithi047 ceph-mon[35327]: Cluster is now healthy 2024-09-19T05:58:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:50 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:58:50.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:50 smithi047 ceph-mon[35327]: fsmap 12 up:standby 2024-09-19T05:58:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:50 smithi179 ceph-mon[35781]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:58:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:50 smithi179 ceph-mon[35781]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:58:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:50 smithi179 ceph-mon[35781]: Cluster is now healthy 2024-09-19T05:58:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:50 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/1687220777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:58:50.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:50 smithi179 ceph-mon[35781]: fsmap 12 up:standby 2024-09-19T05:58:51.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 -- 172.21.15.38:0/3550860183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c106a80 msgr2=0x7f8f8c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:51.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 --2- 172.21.15.38:0/3550860183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c106a80 0x7f8f8c106e60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009930 tx=0x7f8f8802a420 comp rx=0 tx=0).stop 2024-09-19T05:58:51.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 -- 172.21.15.38:0/3550860183 shutdown_connections 2024-09-19T05:58:51.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 --2- 172.21.15.38:0/3550860183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c101420 0x7f8f8c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 --2- 172.21.15.38:0/3550860183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c100a80 0x7f8f8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 --2- 172.21.15.38:0/3550860183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c106a80 0x7f8f8c106e60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 -- 172.21.15.38:0/3550860183 >> 172.21.15.38:0/3550860183 conn(0x7f8f8c0fc7a0 msgr2=0x7f8f8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:51.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 -- 172.21.15.38:0/3550860183 shutdown_connections 2024-09-19T05:58:51.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.322+0000 7f8f93877640 1 -- 172.21.15.38:0/3550860183 wait complete. 2024-09-19T05:58:51.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 Processor -- start 2024-09-19T05:58:51.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 -- start start 2024-09-19T05:58:51.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:51.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c101420 0x7f8f8c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:51.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 0x7f8f8c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110e40 con 0x7f8f8c100a80 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110cc0 con 0x7f8f8c106a80 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f93877640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110fc0 con 0x7f8f8c101420 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f91613640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f91613640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54602/0 (socket says 172.21.15.38:54602) 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.323+0000 7f8f91613640 1 -- 172.21.15.38:0/3436805071 learned_addr learned my addr 172.21.15.38:0/3436805071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f90e12640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c101420 0x7f8f8c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91e14640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 0x7f8f8c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 -- 172.21.15.38:0/3436805071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c101420 msgr2=0x7f8f8c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c101420 0x7f8f8c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 -- 172.21.15.38:0/3436805071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 msgr2=0x7f8f8c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 0x7f8f8c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 -- 172.21.15.38:0/3436805071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19d370 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91e14640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 0x7f8f8c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f91613640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009a50 tx=0x7f8f88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8803f030 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88003ad0 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c19d600 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88035070 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.324+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a4f40 con 0x7f8f8c100a80 2024-09-19T05:58:51.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.325+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c100ee0 con 0x7f8f8c100a80 2024-09-19T05:58:51.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.327+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8f88042030 con 0x7f8f8c100a80 2024-09-19T05:58:51.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.327+0000 7f8f827fc640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 0x7f8f780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:51.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.327+0000 7f8f90e12640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 0x7f8f780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:51.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.327+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9617+0+0 (secure 0 0 0) 0x7f8f880f0a00 con 0x7f8f8c100a80 2024-09-19T05:58:51.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.328+0000 7f8f90e12640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 0x7f8f780a5780 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ff220 tx=0x7f8f7c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:51.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.330+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f880bd1b0 con 0x7f8f8c100a80 2024-09-19T05:58:51.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.508+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f8c10a000 con 0x7f8f8c100a80 2024-09-19T05:58:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.551+0000 7f8f827fc640 1 -- 172.21.15.38:0/3436805071 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f8f880bcab0 con 0x7f8f8c100a80 2024-09-19T05:58:51.553 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 msgr2=0x7f8f780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 0x7f8f780a5780 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ff220 tx=0x7f8f7c00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 msgr2=0x7f8f8c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009a50 tx=0x7f8f88004270 comp rx=0 tx=0).stop 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 shutdown_connections 2024-09-19T05:58:51.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7f8f780a32c0 0x7f8f780a5780 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f8c106a80 0x7f8f8c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f8c101420 0x7f8f8c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 --2- 172.21.15.38:0/3436805071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f8c100a80 0x7f8f8c1983c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.555+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 >> 172.21.15.38:0/3436805071 conn(0x7f8f8c0fc7a0 msgr2=0x7f8f8c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.556+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 shutdown_connections 2024-09-19T05:58:51.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:51.556+0000 7f8f93877640 1 -- 172.21.15.38:0/3436805071 wait complete. 2024-09-19T05:58:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:51 smithi038 ceph-mon[31227]: pgmap v882: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-19T05:58:51.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:51 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:51.902 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:51 smithi047 ceph-mon[35327]: pgmap v882: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-19T05:58:51.902 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:51 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:51 smithi179 ceph-mon[35781]: pgmap v882: 129 pgs: 129 active+clean; 7.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-19T05:58:51.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:51 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:52.249 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T05:58:52.514 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:52.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:52 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:52.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:52 smithi038 ceph-mon[31227]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:58:52.896 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:52 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:52.897 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:52 smithi047 ceph-mon[35327]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:58:52.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:52 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3436805071' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:52.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:52 smithi179 ceph-mon[35781]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- 172.21.15.38:0/2277745736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf4101280 msgr2=0x7feaf410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2277745736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf4101280 0x7feaf410df40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feae4009900 tx=0x7feae402a410 comp rx=0 tx=0).stop 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- 172.21.15.38:0/2277745736 shutdown_connections 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2277745736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf410e480 0x7feaf4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2277745736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf4101280 0x7feaf410df40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2277745736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf41008d0 0x7feaf4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- 172.21.15.38:0/2277745736 >> 172.21.15.38:0/2277745736 conn(0x7feaf40fc7a0 msgr2=0x7feaf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- 172.21.15.38:0/2277745736 shutdown_connections 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- 172.21.15.38:0/2277745736 wait complete. 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 Processor -- start 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.041+0000 7feafbcd9640 1 -- start start 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 0x7feaf419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf410e480 0x7feaf419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41141b0 con 0x7feaf41008d0 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4114030 con 0x7feaf4101280 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafbcd9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feaf4114330 con 0x7feaf410e480 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:53.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54630/0 (socket says 172.21.15.38:54630) 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 -- 172.21.15.38:0/2994801804 learned_addr learned my addr 172.21.15.38:0/2994801804 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feafa276640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf410e480 0x7feaf419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9274640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 0x7feaf419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 -- 172.21.15.38:0/2994801804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf410e480 msgr2=0x7feaf419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf410e480 0x7feaf419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 -- 172.21.15.38:0/2994801804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 msgr2=0x7feaf419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 0x7feaf419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 -- 172.21.15.38:0/2994801804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf419f6a0 con 0x7feaf41008d0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9274640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 0x7feaf419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaf9a75640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf000d770 tx=0x7feaf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.042+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0017070 con 0x7feaf41008d0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.043+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0005500 con 0x7feaf41008d0 2024-09-19T05:58:53.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.043+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf419f990 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.043+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0004e00 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.043+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf41a7270 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.043+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf4100cb0 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.045+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7feaf0004fa0 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.045+0000 7feaeaffd640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 0x7feae00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.046+0000 7feaf9274640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 0x7feae00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.046+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9084+0+0 (secure 0 0 0) 0x7feaf00cfbb0 con 0x7feaf41008d0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.046+0000 7feaf9274640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 0x7feae00a5780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaf410ed80 tx=0x7feae4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:53.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.049+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feaf009c5a0 con 0x7feaf41008d0 2024-09-19T05:58:53.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.228+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7feaf419ba50 con 0x7feaf41008d0 2024-09-19T05:58:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.580+0000 7feaeaffd640 1 -- 172.21.15.38:0/2994801804 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7feaf419ba50 con 0x7feaf41008d0 2024-09-19T05:58:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-19T05:58:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 msgr2=0x7feae00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 0x7feae00a5780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaf410ed80 tx=0x7feae4005a20 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 msgr2=0x7feaf419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf000d770 tx=0x7feaf000dc30 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 shutdown_connections 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7feae00a32c0 0x7feae00a5780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf410e480 0x7feaf419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feaf4101280 0x7feaf419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 --2- 172.21.15.38:0/2994801804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf41008d0 0x7feaf419a6f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 >> 172.21.15.38:0/2994801804 conn(0x7feaf40fc7a0 msgr2=0x7feaf40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 shutdown_connections 2024-09-19T05:58:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:53.583+0000 7feafbcd9640 1 -- 172.21.15.38:0/2994801804 wait complete. 2024-09-19T05:58:53.893 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:53 smithi038 ceph-mon[31227]: pgmap v884: 97 pgs: 97 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:53.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:53 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:53 smithi179 ceph-mon[35781]: pgmap v884: 97 pgs: 97 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:53.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:53 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:53.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:53 smithi047 ceph-mon[35327]: pgmap v884: 97 pgs: 97 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:53.976 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:53 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:54.267 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-19T05:58:54.528 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:58:54.846 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:54 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:54.847 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:54 smithi038 ceph-mon[31227]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:58:54.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:54 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:54.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:54 smithi179 ceph-mon[35781]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:58:54.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:54 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/2994801804' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:54.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:54 smithi047 ceph-mon[35327]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:58:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- 172.21.15.38:0/340391200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c100a80 msgr2=0x7fea4c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 --2- 172.21.15.38:0/340391200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c100a80 0x7fea4c100ee0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea40009930 tx=0x7fea4002a420 comp rx=0 tx=0).stop 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- 172.21.15.38:0/340391200 shutdown_connections 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 --2- 172.21.15.38:0/340391200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c101420 0x7fea4c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 --2- 172.21.15.38:0/340391200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c100a80 0x7fea4c100ee0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 --2- 172.21.15.38:0/340391200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 0x7fea4c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- 172.21.15.38:0/340391200 >> 172.21.15.38:0/340391200 conn(0x7fea4c0fc7a0 msgr2=0x7fea4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- 172.21.15.38:0/340391200 shutdown_connections 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- 172.21.15.38:0/340391200 wait complete. 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 Processor -- start 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.041+0000 7fea5275a640 1 -- start start 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c100a80 0x7fea4c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 0x7fea4c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c110e80 con 0x7fea4c101420 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c110d00 con 0x7fea4c106a80 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea5275a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c111000 con 0x7fea4c100a80 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54640/0 (socket says 172.21.15.38:54640) 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 -- 172.21.15.38:0/3800787263 learned_addr learned my addr 172.21.15.38:0/3800787263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4bfff640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c100a80 0x7fea4c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea50cf7640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 0x7fea4c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 -- 172.21.15.38:0/3800787263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c100a80 msgr2=0x7fea4c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c100a80 0x7fea4c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 -- 172.21.15.38:0/3800787263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 msgr2=0x7fea4c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 0x7fea4c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.042+0000 7fea4b7fe640 1 -- 172.21.15.38:0/3800787263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c19f5f0 con 0x7fea4c101420 2024-09-19T05:58:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea4b7fe640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4003f030 con 0x7fea4c101420 2024-09-19T05:58:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40003ad0 con 0x7fea4c101420 2024-09-19T05:58:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c19f880 con 0x7fea4c101420 2024-09-19T05:58:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40035070 con 0x7fea4c101420 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.043+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1a71c0 con 0x7fea4c101420 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.044+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c1021c0 con 0x7fea4c101420 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.045+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fea40042030 con 0x7fea4c101420 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.046+0000 7fea497fa640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 0x7fea340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.046+0000 7fea4bfff640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 0x7fea340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.046+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8496+0+0 (secure 0 0 0) 0x7fea400efe50 con 0x7fea4c101420 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.047+0000 7fea4bfff640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 0x7fea340a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0058c0 tx=0x7fea3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:58:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.051+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea400bc420 con 0x7fea4c101420 2024-09-19T05:58:55.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.226+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea4c1a7390 con 0x7fea4c101420 2024-09-19T05:58:55.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.593+0000 7fea497fa640 1 -- 172.21.15.38:0/3800787263 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) ==== 178+0+0 (secure 0 0 0) 0x7fea400c0720 con 0x7fea4c101420 2024-09-19T05:58:55.595 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' removed 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 msgr2=0x7fea340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 0x7fea340a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0058c0 tx=0x7fea3c00a2d0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 msgr2=0x7fea4c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea40004270 comp rx=0 tx=0).stop 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 shutdown_connections 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:6800/4137587885,v1:172.21.15.38:6801/4137587885] conn(0x7fea340a3240 0x7fea340a5700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c106a80 0x7fea4c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea4c101420 0x7fea4c19ab80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 --2- 172.21.15.38:0/3800787263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c100a80 0x7fea4c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:58:55.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 >> 172.21.15.38:0/3800787263 conn(0x7fea4c0fc7a0 msgr2=0x7fea4c10c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:58:55.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 shutdown_connections 2024-09-19T05:58:55.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-19T05:58:55.595+0000 7fea5275a640 1 -- 172.21.15.38:0/3800787263 wait complete. 2024-09-19T05:58:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:55 smithi038 ceph-mon[31227]: pgmap v886: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:55.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:55 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:55 smithi179 ceph-mon[35781]: pgmap v886: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:55.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:55 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:55 smithi047 ceph-mon[35327]: pgmap v886: 33 pgs: 33 active+clean; 577 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:58:55.975 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:55 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:58:56.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:58:56.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:58:56.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:58:56.444 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T05:58:56.444 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:58:56.461 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:58:56.479 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:58:56.502 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T05:58:56.502 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T05:58:56.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a 2024-09-19T05:58:56.611 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 systemd[1]: Stopping Ceph mon.a for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:58:56.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:56 smithi179 ceph-mon[35781]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:56.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:56 smithi179 ceph-mon[35781]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:58:56.725 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:56 smithi047 ceph-mon[35327]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:56.726 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:56 smithi047 ceph-mon[35327]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:58:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 ceph-mon[31227]: from='client.? 172.21.15.38:0/3800787263' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:58:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 ceph-mon[31227]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:58:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a[31223]: 2024-09-19T05:58:56.816+0000 7fd2d608a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:58:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a[31223]: 2024-09-19T05:58:56.816+0000 7fd2d608a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T05:58:56.894 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:56 smithi038 podman[97858]: 2024-09-19 05:58:56.857425584 +0000 UTC m=+0.155945124 container died edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:58:57.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:57 smithi038 podman[97858]: 2024-09-19 05:58:57.241923564 +0000 UTC m=+0.540443102 container cleanup edf8305fb6ee027409e73bf84109759af799712729d74297aedf620441bdfb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T05:58:57.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 19 05:58:57 smithi038 bash[97858]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-a 2024-09-19T05:58:57.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T05:58:57.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:58:57.716 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T05:58:57.716 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T05:58:57.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.b 2024-09-19T05:58:58.035 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:57 smithi047 systemd[1]: Stopping Ceph mon.b for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:58:58.475 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b[35323]: 2024-09-19T05:58:58.033+0000 7f36f1bf8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:58:58.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b[35323]: 2024-09-19T05:58:58.033+0000 7f36f1bf8640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:58:58.476 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:58 smithi047 podman[70231]: 2024-09-19 05:58:58.197567949 +0000 UTC m=+0.281969600 container died caf1f61d59912133bf96e5b81a0dd0fed29307c54733543a07ea4c80cd7376df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T05:58:58.963 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:58 smithi047 podman[70231]: 2024-09-19 05:58:58.609551679 +0000 UTC m=+0.693953284 container cleanup caf1f61d59912133bf96e5b81a0dd0fed29307c54733543a07ea4c80cd7376df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T05:58:58.963 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 19 05:58:58 smithi047 bash[70231]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-b 2024-09-19T05:58:59.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T05:58:59.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:58:59.128 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T05:58:59.128 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T05:58:59.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.c 2024-09-19T05:58:59.413 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 systemd[1]: Stopping Ceph mon.c for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:58:59.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-c[35777]: 2024-09-19T05:58:59.450+0000 7fe6fc2d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:58:59.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-c[35777]: 2024-09-19T05:58:59.450+0000 7fe6fc2d6640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:58:59.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 podman[72137]: 2024-09-19 05:58:59.542257759 +0000 UTC m=+0.214608262 container died b347b383754a4c37aecafbdddc1fb0ae6aea56df8b37be21587e3ecb8577138f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T05:59:00.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 podman[72137]: 2024-09-19 05:58:59.933801407 +0000 UTC m=+0.606151914 container cleanup b347b383754a4c37aecafbdddc1fb0ae6aea56df8b37be21587e3ecb8577138f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:59:00.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Sep 19 05:58:59 smithi179 bash[72137]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mon-c 2024-09-19T05:59:00.350 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T05:59:00.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:00.396 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T05:59:00.396 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T05:59:00.397 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x 2024-09-19T05:59:00.732 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:59:00 smithi038 systemd[1]: Stopping Ceph mgr.x for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:01.058 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:59:00 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x[31452]: 2024-09-19T05:59:00.730+0000 7f39634d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:01.059 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:59:00 smithi038 podman[97980]: 2024-09-19 05:59:00.761731667 +0000 UTC m=+0.160277897 container died f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:59:01.393 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:59:01 smithi038 podman[97980]: 2024-09-19 05:59:01.165346952 +0000 UTC m=+0.563893169 container cleanup f594ec0a8f311caf22c42001384de285d7467a873217c8e95a78a4df5c86e057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T05:59:01.394 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 19 05:59:01 smithi038 bash[97980]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-x 2024-09-19T05:59:01.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T05:59:01.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:01.643 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T05:59:01.643 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T05:59:01.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.y 2024-09-19T05:59:01.961 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:59:01 smithi047 systemd[1]: Stopping Ceph mgr.y for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:02.225 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:59:01 smithi047 podman[70355]: 2024-09-19 05:59:01.960556443 +0000 UTC m=+0.114667041 container died e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T05:59:02.475 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:59:02 smithi047 podman[70355]: 2024-09-19 05:59:02.314944201 +0000 UTC m=+0.469054829 container cleanup e7c91e3aa3f15dc71ec673c4c814c4d43dbcc4a0019d9db38fea599b47bc87c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T05:59:02.476 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 19 05:59:02 smithi047 bash[70355]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-y 2024-09-19T05:59:02.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T05:59:02.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:02.787 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T05:59:02.787 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T05:59:02.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.z 2024-09-19T05:59:03.127 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:02 smithi179 systemd[1]: Stopping Ceph mgr.z for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:03.417 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:03 smithi179 podman[72273]: 2024-09-19 05:59:03.126541361 +0000 UTC m=+0.145662125 container died 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T05:59:03.697 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:03 smithi179 podman[72273]: 2024-09-19 05:59:03.531630949 +0000 UTC m=+0.550751722 container cleanup 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T05:59:03.697 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:03 smithi179 bash[72273]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z 2024-09-19T05:59:04.001 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:03 smithi179 podman[72285]: 2024-09-19 05:59:03.750219102 +0000 UTC m=+0.615642702 container remove 829bcaf4e8365d68daacdf2e337d972927b2d4b6a3b774ec6305b565f47ea2ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T05:59:04.002 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:03 smithi179 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T05:59:04.002 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:04 smithi179 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T05:59:04.002 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Sep 19 05:59:04 smithi179 systemd[1]: Stopped Ceph mgr.z for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T05:59:04.004 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T05:59:04.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:04.039 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T05:59:04.039 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T05:59:04.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.0 2024-09-19T05:59:04.355 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:04 smithi038 systemd[1]: Stopping Ceph osd.0 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:04.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:59:04.353+0000 7f8650e7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:04.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:59:04.353+0000 7f8650e7e640 -1 osd.0 95 *** Got signal Terminated *** 2024-09-19T05:59:04.644 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0[42831]: 2024-09-19T05:59:04.353+0000 7f8650e7e640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:09.948 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:09 smithi038 podman[98102]: 2024-09-19 05:59:09.66574908 +0000 UTC m=+5.426431633 container died 042cf02423679eefe94c96c8744b96215b89bf193ab2702b3b423a1f60c605b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:59:10.394 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:10 smithi038 podman[98102]: 2024-09-19 05:59:10.138623633 +0000 UTC m=+5.899306213 container cleanup 042cf02423679eefe94c96c8744b96215b89bf193ab2702b3b423a1f60c605b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T05:59:10.394 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:10 smithi038 bash[98102]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0 2024-09-19T05:59:10.773 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:10 smithi038 podman[98117]: 2024-09-19 05:59:10.432121493 +0000 UTC m=+0.763877513 container remove 042cf02423679eefe94c96c8744b96215b89bf193ab2702b3b423a1f60c605b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T05:59:11.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:10 smithi038 podman[98209]: 2024-09-19 05:59:10.674869844 +0000 UTC m=+0.017368733 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:11.144 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:10 smithi038 podman[98209]: 2024-09-19 05:59:10.866668508 +0000 UTC m=+0.209167387 container create ffd2ecfd1daade147d7914e1d58acb78b8c4ffbe4b83f377c2b55a9fe3afac9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:59:11.435 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:11 smithi038 podman[98209]: 2024-09-19 05:59:11.144787067 +0000 UTC m=+0.487285963 container init ffd2ecfd1daade147d7914e1d58acb78b8c4ffbe4b83f377c2b55a9fe3afac9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:11.435 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:11 smithi038 podman[98209]: 2024-09-19 05:59:11.148614577 +0000 UTC m=+0.491113459 container start ffd2ecfd1daade147d7914e1d58acb78b8c4ffbe4b83f377c2b55a9fe3afac9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T05:59:11.435 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:11 smithi038 podman[98209]: 2024-09-19 05:59:11.236791563 +0000 UTC m=+0.579290443 container attach ffd2ecfd1daade147d7914e1d58acb78b8c4ffbe4b83f377c2b55a9fe3afac9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T05:59:11.435 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 19 05:59:11 smithi038 podman[98209]: 2024-09-19 05:59:11.351730835 +0000 UTC m=+0.694229714 container died ffd2ecfd1daade147d7914e1d58acb78b8c4ffbe4b83f377c2b55a9fe3afac9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T05:59:11.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T05:59:12.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:12.038 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T05:59:12.038 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T05:59:12.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.1 2024-09-19T05:59:12.354 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:12 smithi047 systemd[1]: Stopping Ceph osd.1 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:12.725 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:59:12.352+0000 7fd00142e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:12.726 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:59:12.352+0000 7fd00142e640 -1 osd.1 95 *** Got signal Terminated *** 2024-09-19T05:59:12.726 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1[39796]: 2024-09-19T05:59:12.352+0000 7fd00142e640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:17.988 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:17 smithi047 podman[70482]: 2024-09-19 05:59:17.727539722 +0000 UTC m=+5.491827002 container died b379888bff0b047c9507e30969d5af50773d355e6a17822f04963ef9d07a19c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T05:59:18.322 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:18 smithi047 podman[70482]: 2024-09-19 05:59:18.069761915 +0000 UTC m=+5.834049191 container cleanup b379888bff0b047c9507e30969d5af50773d355e6a17822f04963ef9d07a19c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T05:59:18.322 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:18 smithi047 bash[70482]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1 2024-09-19T05:59:18.725 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:18 smithi047 podman[70502]: 2024-09-19 05:59:18.321132325 +0000 UTC m=+0.593627856 container remove b379888bff0b047c9507e30969d5af50773d355e6a17822f04963ef9d07a19c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T05:59:19.042 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:18 smithi047 podman[70589]: 2024-09-19 05:59:18.64627978 +0000 UTC m=+0.015836715 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:19.043 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:18 smithi047 podman[70589]: 2024-09-19 05:59:18.780790686 +0000 UTC m=+0.150347661 container create 3caea056a1ebe3d58e61cf75e589c39c1d16fd0e4952542da6bf55acdb9d364c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T05:59:19.043 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:19 smithi047 podman[70589]: 2024-09-19 05:59:19.041588958 +0000 UTC m=+0.411145887 container init 3caea056a1ebe3d58e61cf75e589c39c1d16fd0e4952542da6bf55acdb9d364c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T05:59:19.327 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:19 smithi047 podman[70589]: 2024-09-19 05:59:19.045124145 +0000 UTC m=+0.414681072 container start 3caea056a1ebe3d58e61cf75e589c39c1d16fd0e4952542da6bf55acdb9d364c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T05:59:19.327 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:19 smithi047 podman[70589]: 2024-09-19 05:59:19.141891593 +0000 UTC m=+0.511448528 container attach 3caea056a1ebe3d58e61cf75e589c39c1d16fd0e4952542da6bf55acdb9d364c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T05:59:19.327 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 19 05:59:19 smithi047 podman[70589]: 2024-09-19 05:59:19.240673245 +0000 UTC m=+0.610230171 container died 3caea056a1ebe3d58e61cf75e589c39c1d16fd0e4952542da6bf55acdb9d364c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T05:59:19.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T05:59:19.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:19.893 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T05:59:19.894 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T05:59:19.894 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.2 2024-09-19T05:59:20.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:19 smithi179 systemd[1]: Stopping Ceph osd.2 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:20.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:59:20.202+0000 7efc50518640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:20.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:59:20.202+0000 7efc50518640 -1 osd.2 95 *** Got signal Terminated *** 2024-09-19T05:59:20.697 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2[40730]: 2024-09-19T05:59:20.202+0000 7efc50518640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:25.847 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:25 smithi179 podman[72401]: 2024-09-19 05:59:25.546149808 +0000 UTC m=+5.463533978 container died 65bd0334b67ad79f436cf7fd99357b7e2463b09f789bf9696c1289dd95e0142c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:59:26.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:25 smithi179 podman[72401]: 2024-09-19 05:59:25.936258092 +0000 UTC m=+5.853642250 container cleanup 65bd0334b67ad79f436cf7fd99357b7e2463b09f789bf9696c1289dd95e0142c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:59:26.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:25 smithi179 bash[72401]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2 2024-09-19T05:59:26.557 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72418]: 2024-09-19 05:59:26.204651263 +0000 UTC m=+0.652888686 container remove 65bd0334b67ad79f436cf7fd99357b7e2463b09f789bf9696c1289dd95e0142c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T05:59:26.851 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72509]: 2024-09-19 05:59:26.458590535 +0000 UTC m=+0.015430486 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:26.851 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72509]: 2024-09-19 05:59:26.580569198 +0000 UTC m=+0.137409148 container create 38c0916091f4b7bf398f011f77101a977156598c589e937cd8d6cba4e85fe41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:26.851 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72509]: 2024-09-19 05:59:26.849884435 +0000 UTC m=+0.406724388 container init 38c0916091f4b7bf398f011f77101a977156598c589e937cd8d6cba4e85fe41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T05:59:27.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72509]: 2024-09-19 05:59:26.853659227 +0000 UTC m=+0.410499179 container start 38c0916091f4b7bf398f011f77101a977156598c589e937cd8d6cba4e85fe41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T05:59:27.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:26 smithi179 podman[72509]: 2024-09-19 05:59:26.958389743 +0000 UTC m=+0.515229716 container attach 38c0916091f4b7bf398f011f77101a977156598c589e937cd8d6cba4e85fe41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T05:59:27.197 INFO:journalctl@ceph.osd.2.smithi179.stdout:Sep 19 05:59:27 smithi179 podman[72509]: 2024-09-19 05:59:27.046695715 +0000 UTC m=+0.603535669 container died 38c0916091f4b7bf398f011f77101a977156598c589e937cd8d6cba4e85fe41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T05:59:27.700 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T05:59:27.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:27.738 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T05:59:27.738 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T05:59:27.739 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.3 2024-09-19T05:59:28.054 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:27 smithi038 systemd[1]: Stopping Ceph osd.3 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:28.394 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:28 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:59:28.051+0000 7f219855c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:28.394 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:28 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:59:28.051+0000 7f219855c640 -1 osd.3 95 *** Got signal Terminated *** 2024-09-19T05:59:28.394 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:28 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:59:28.051+0000 7f219855c640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:32.727 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:32.441+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:33.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:32 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3[49439]: 2024-09-19T05:59:32.725+0000 7f2190b6a640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:07.257679+0000 front 2024-09-19T05:59:07.257799+0000 (oldest deadline 2024-09-19T05:59:31.858317+0000) 2024-09-19T05:59:33.297 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:33.026+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:33.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:33.294+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:33.780 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:33 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:33.409+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:33.781 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:33 smithi038 podman[98323]: 2024-09-19 05:59:33.476579901 +0000 UTC m=+5.539330983 container died 40ba975bb0432facda2ed6c312f3b1b163aa20240b81708617c72160a55c48d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T05:59:33.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:33 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:33.497+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:34.143 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:33 smithi038 podman[98323]: 2024-09-19 05:59:33.970324204 +0000 UTC m=+6.033075286 container cleanup 40ba975bb0432facda2ed6c312f3b1b163aa20240b81708617c72160a55c48d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T05:59:34.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:33 smithi038 bash[98323]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3 2024-09-19T05:59:34.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:33 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:33.992+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:34.255 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:33.988+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:34.552 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98338]: 2024-09-19 05:59:34.221932077 +0000 UTC m=+0.739829380 container remove 40ba975bb0432facda2ed6c312f3b1b163aa20240b81708617c72160a55c48d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T05:59:34.552 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:34 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:34.422+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:34.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:34.253+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:34.725 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:34 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:34.471+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:34.845 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98427]: 2024-09-19 05:59:34.453637203 +0000 UTC m=+0.016331561 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:34.845 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98427]: 2024-09-19 05:59:34.581373982 +0000 UTC m=+0.144068384 container create b0558ac590e570223f35570540ade4e5a8f05ad875d88ab84c3707dbabaf89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T05:59:35.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98427]: 2024-09-19 05:59:34.844139636 +0000 UTC m=+0.406834002 container init b0558ac590e570223f35570540ade4e5a8f05ad875d88ab84c3707dbabaf89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:59:35.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98427]: 2024-09-19 05:59:34.848032606 +0000 UTC m=+0.410726964 container start b0558ac590e570223f35570540ade4e5a8f05ad875d88ab84c3707dbabaf89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T05:59:35.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:34 smithi038 podman[98427]: 2024-09-19 05:59:34.952287812 +0000 UTC m=+0.514982186 container attach b0558ac590e570223f35570540ade4e5a8f05ad875d88ab84c3707dbabaf89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:35.144 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 19 05:59:35 smithi038 podman[98427]: 2024-09-19 05:59:35.039058217 +0000 UTC m=+0.601752577 container died b0558ac590e570223f35570540ade4e5a8f05ad875d88ab84c3707dbabaf89fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T05:59:35.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:35 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:35.030+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:35.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:35.260+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:35.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:35.033+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:35.643 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:35 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:35.378+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:35.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:35 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:35.448+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:35.825 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T05:59:35.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:35.865 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T05:59:35.865 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T05:59:35.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.4 2024-09-19T05:59:36.143 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:36 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:36.027+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:36.226 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:35 smithi047 systemd[1]: Stopping Ceph osd.4 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:36.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:36.024+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:36.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:36.269+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:36.644 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:36 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:36.418+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:36.725 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:36 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:36.478+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:36.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:36 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:36.299+0000 7fb2a62e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:36.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:36 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:36.299+0000 7fb2a62e6640 -1 osd.4 95 *** Got signal Terminated *** 2024-09-19T05:59:36.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:36 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:36.299+0000 7fb2a62e6640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:37.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:37 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:37.040+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:37.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:37 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:37.074+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:37.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:37 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:37.243+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:37.893 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:37 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:37.466+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:37.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:37 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:37.507+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:38.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:38 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:38.073+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:38.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:38 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:38.073+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:38.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:38 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:38.262+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:38.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:38 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:38.026+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:38.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:38 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:38.378+0000 7fb2a28fc640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.342621+0000 front 2024-09-19T05:59:13.342641+0000 (oldest deadline 2024-09-19T05:59:38.042857+0000) 2024-09-19T05:59:38.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:38 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:38.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:38.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:38 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:38.468+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:38.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:38 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:38.468+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:39.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:39.037+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:39.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:39.037+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:39.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:39 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:39.218+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:39.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:38 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:38.985+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:39.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:38 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:38.985+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:39.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:39 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:39.426+0000 7fb2a28fc640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.342621+0000 front 2024-09-19T05:59:13.342641+0000 (oldest deadline 2024-09-19T05:59:38.042857+0000) 2024-09-19T05:59:39.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:39 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:39.563+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:39.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:39.498+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:39.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:39.498+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:40.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:40.066+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:40.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:40.066+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:40.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:39 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:39.991+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:40.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:39 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:39.991+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:40.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:40 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:40.199+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:40.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:40 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:40.234+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:40.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:40 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:40.234+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:40.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:40 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4[46224]: 2024-09-19T05:59:40.453+0000 7fb2a28fc640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.342621+0000 front 2024-09-19T05:59:13.342641+0000 (oldest deadline 2024-09-19T05:59:38.042857+0000) 2024-09-19T05:59:40.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:40 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:40.612+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:40.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:40.530+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:40.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:40 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:40.530+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:41.241 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:40 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:40.958+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:41.241 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:40 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:40.958+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:41.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:41.078+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:41.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:41.078+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:41.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:41 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:41.238+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:41.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:41 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:41.274+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:41.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:41 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:41.274+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:41.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:41.525+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:41.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:41 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:41.525+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:41.975 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:41 smithi047 podman[70715]: 2024-09-19 05:59:41.774950456 +0000 UTC m=+5.708155147 container died fd63f68774baa50d967d98ad994bdf80c137e31754feed7199d66e654aca4711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:41.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:41 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:41.639+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:42.268 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:41 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:41.988+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:42.268 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:41 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:41.988+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:42.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:42.109+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:42.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:42.109+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:42.409 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:42 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:42.162+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:42.409 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70715]: 2024-09-19 05:59:42.157457394 +0000 UTC m=+6.090662071 container cleanup fd63f68774baa50d967d98ad994bdf80c137e31754feed7199d66e654aca4711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:59:42.409 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 bash[70715]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4 2024-09-19T05:59:42.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:42 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:42.266+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:42.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:42 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:42.271+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:42.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:42 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:42.271+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:42.726 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70732]: 2024-09-19 05:59:42.408630506 +0000 UTC m=+0.629449304 container remove fd63f68774baa50d967d98ad994bdf80c137e31754feed7199d66e654aca4711 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:59:42.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:42 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:42.594+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:42.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:42 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:42.594+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:42.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:42.527+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:42.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:42 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:42.527+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:42.983 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70819]: 2024-09-19 05:59:42.632436727 +0000 UTC m=+0.015863733 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:42.983 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70819]: 2024-09-19 05:59:42.734434313 +0000 UTC m=+0.117861322 container create db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T05:59:42.984 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70819]: 2024-09-19 05:59:42.979024081 +0000 UTC m=+0.362451088 container init db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:59:42.984 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:42 smithi047 podman[70819]: 2024-09-19 05:59:42.982313654 +0000 UTC m=+0.365740660 container start db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T05:59:43.252 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:42 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:42.967+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:43.253 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:42 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:42.967+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:43.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:43.065+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:43.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:43.065+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:43.476 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:43 smithi047 podman[70819]: 2024-09-19 05:59:43.087430451 +0000 UTC m=+0.470857460 container attach db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T05:59:43.476 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:43 smithi047 podman[70819]: 2024-09-19 05:59:43.177522078 +0000 UTC m=+0.560949084 container died db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T05:59:43.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:43 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:43.133+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:43.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:43.293+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:43.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:43.251+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:43.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:43.251+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:43.790 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 19 05:59:43 smithi047 podman[70843]: 2024-09-19 05:59:43.766219015 +0000 UTC m=+0.582685625 container remove db6a7624900aa4f89ab5853bfa37f126b033e2048216c8a5497c9fd99d7c099f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:59:43.790 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:43 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:43.631+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:43.790 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:43 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:43.631+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:43.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T05:59:43.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:43.836 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T05:59:43.836 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T05:59:43.836 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.5 2024-09-19T05:59:43.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:43.488+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:43.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:43 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:43.488+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:44.159 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:43 smithi179 systemd[1]: Stopping Ceph osd.5 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:44.159 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:43.948+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:44.159 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:43 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:43.948+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:44.159 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:44.157+0000 7f978efb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:44.159 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:44.157+0000 7f978efb1640 -1 osd.5 95 *** Got signal Terminated *** 2024-09-19T05:59:44.160 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:44.157+0000 7f978efb1640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:44.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:44.031+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:44.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:44.031+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:44.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:44.338+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:44.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:44.220+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:44.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:44.220+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:44.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:44 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:44.139+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:44.896 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:44.482+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:44.896 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:44.482+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:44.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:44 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:44.650+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:44.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:44 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:44.650+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:45.197 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:44.933+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:45.197 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:44 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:44.933+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:45.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:45 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:45.176+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:45.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:45 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:45.176+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:45.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:44.989+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:45.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:44.989+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:45.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:45 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:45.107+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:45.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:45 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:45.322+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:45.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:45 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:45.461+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:45.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:45 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:45.462+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:45.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:45 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:45.665+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:45.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:45 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:45.665+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:46.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:45 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:45.979+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:46.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:45 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:45.979+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:46.322 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:46 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:46.209+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:46.322 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:46 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:46.209+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:46.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:46.019+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:46.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:46.019+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:46.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:46 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:46.066+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:46.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:46 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:46.320+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:46.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:46.504+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:46.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:46 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:46.504+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:46.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:46 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:46.714+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:46.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:46 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:46.714+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:47.347 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:46 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:46.992+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:47.347 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:46 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:46.992+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:47.348 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:47.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:47.348 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:47.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:47.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:47.051+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:47.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:47.051+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:47.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:47.049+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:47.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:47.049+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:47.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:47.345+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:47.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:47.454+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:47.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:47 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:47.454+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:47.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:47.674+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:47.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:47 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:47.674+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:48.386 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:47.960+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:48.387 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:47 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:47.960+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:48.387 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:48.177+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:48.387 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:48.177+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:48.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:48.088+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:48.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:48.088+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:48.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:48 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:48.028+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:48.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:48 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:48.028+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:48.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:48.385+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:48.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:48.477+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:48.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:48.477+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:48.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:48.477+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:48.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:48 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:48.678+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:48.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:48 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:48.678+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:48.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:48 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:48.678+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:49.197 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:48.912+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:06.549696+0000 front 2024-09-19T05:59:06.549748+0000 (oldest deadline 2024-09-19T05:59:32.449742+0000) 2024-09-19T05:59:49.197 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:48.912+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:15.950557+0000 front 2024-09-19T05:59:15.950575+0000 (oldest deadline 2024-09-19T05:59:38.850663+0000) 2024-09-19T05:59:49.198 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:48 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5[47603]: 2024-09-19T05:59:48.912+0000 7f978b5c7640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.951492+0000 front 2024-09-19T05:59:22.951559+0000 (oldest deadline 2024-09-19T05:59:48.251624+0000) 2024-09-19T05:59:49.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:49 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:49.132+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:49.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:49 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:49.132+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:49.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:49 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:49.132+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:49.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:49.123+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:49.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:49.123+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:49.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:49.123+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:49.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:49.004+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:49.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:49.004+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:49.697 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:49 smithi179 podman[72622]: 2024-09-19 05:59:49.498211912 +0000 UTC m=+5.464672755 container died 463879aec1e46a8b09e2329347be4bcfea23c4769f8f66f81fc6179c71d0f8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:49.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:49 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:49.408+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:49.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:49 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:49.408+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:49.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:49.527+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:49.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:49.527+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:49.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:49 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:49.527+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:49.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:49.669+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:49.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:49.669+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:49.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:49.669+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:50.104 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:49 smithi179 podman[72622]: 2024-09-19 05:59:49.874400451 +0000 UTC m=+5.840861293 container cleanup 463879aec1e46a8b09e2329347be4bcfea23c4769f8f66f81fc6179c71d0f8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:59:50.104 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:49 smithi179 bash[72622]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5 2024-09-19T05:59:50.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:50.094+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:50.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:50.094+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:50.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:50.094+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:50.447 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72639]: 2024-09-19 05:59:50.167425935 +0000 UTC m=+0.665704330 container remove 463879aec1e46a8b09e2329347be4bcfea23c4769f8f66f81fc6179c71d0f8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:59:50.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:50 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:50.101+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:50.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:50 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:50.101+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:50.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:50 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:50.101+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:50.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:49.981+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:50.475 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:49 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:49.981+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:50.763 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:50 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:50.457+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:50.763 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:50 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:50.457+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:50.764 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72724]: 2024-09-19 05:59:50.388926832 +0000 UTC m=+0.015933216 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:50.764 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72724]: 2024-09-19 05:59:50.501949392 +0000 UTC m=+0.128955805 container create ced4fed62bc64e22aeeee215b778143f0edd03ed94ad2bcb2bb78a60ce58afdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:59:50.764 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72724]: 2024-09-19 05:59:50.762741518 +0000 UTC m=+0.389747889 container init ced4fed62bc64e22aeeee215b778143f0edd03ed94ad2bcb2bb78a60ce58afdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T05:59:50.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:50.517+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:50.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:50.517+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:50.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:50 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:50.517+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:50.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:50 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:50.663+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:50.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:50 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:50.663+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:50.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:50 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:50.663+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:51.093 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72724]: 2024-09-19 05:59:50.765960256 +0000 UTC m=+0.392966627 container start ced4fed62bc64e22aeeee215b778143f0edd03ed94ad2bcb2bb78a60ce58afdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:59:51.093 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72724]: 2024-09-19 05:59:50.862762589 +0000 UTC m=+0.489768952 container attach ced4fed62bc64e22aeeee215b778143f0edd03ed94ad2bcb2bb78a60ce58afdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T05:59:51.093 INFO:journalctl@ceph.osd.5.smithi179.stdout:Sep 19 05:59:50 smithi179 podman[72747]: 2024-09-19 05:59:50.978786192 +0000 UTC m=+0.016391129 container died ced4fed62bc64e22aeeee215b778143f0edd03ed94ad2bcb2bb78a60ce58afdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T05:59:51.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:51.052+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:51.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:51.052+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:51.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:51.052+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:51.408 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:51.090+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:51.408 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:51.090+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:51.409 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:51.090+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:51.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:50 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:50.980+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:51.476 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:50 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:50.980+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:51.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:51.452+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:51.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:51 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:51.452+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:51.739 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T05:59:51.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:51.785 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T05:59:51.785 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T05:59:51.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.6 2024-09-19T05:59:51.813 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:51.500+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:51.813 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:51.500+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:51.814 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:51 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:51.500+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:51.949 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:51 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:51.679+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:51.949 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:51 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:51.679+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:51.950 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:51 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:51.679+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:52.121 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:51 smithi038 systemd[1]: Stopping Ceph osd.6 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T05:59:52.121 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.119+0000 7ff7573ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:59:52.121 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:52.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:52.121 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:52.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:52.121 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:52.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:52.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:51 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:51.947+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:52.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:51 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:51.947+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:52.393 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.119+0000 7ff7573ab640 -1 osd.6 95 *** Got signal Terminated *** 2024-09-19T05:59:52.394 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.119+0000 7ff7573ab640 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:59:52.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:52 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:52.073+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:52.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:52 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:52.073+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:52.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:52 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:52.073+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:52.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.490+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:52.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.490+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:52.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:52.490+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:52.947 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:52 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:52.629+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:52.947 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:52 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:52.629+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:52.947 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:52 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:52.629+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:52.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:52 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:52.452+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:52.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:52 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:52.452+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:53.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:52 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:52.945+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:53.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:52 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:52.945+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:53.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:53.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:53.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:53.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:53.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:53.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:53.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:53.024+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:53.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:53.024+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:53.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:53.024+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:53.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:53.473+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:53.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:53.473+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:53.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:53 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:53.473+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:53.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:53.481+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:53.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:53 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:53.481+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:53.957 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:53 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:53.607+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:53.957 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:53 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:53.607+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:53.957 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:53 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:53.607+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:54.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:53 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:53.956+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:54.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:53 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:53.956+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:54.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:54.075+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:54.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:54.075+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:54.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:54.075+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:54.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.011+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:54.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.011+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:54.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.011+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:54.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:54.465+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:54.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:54.465+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:54.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:54 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:54.465+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:54.916 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:54.653+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:54.916 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:54.653+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:54.916 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:54.653+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:54.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:54.478+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:54.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:54.478+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:55.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:54.915+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:55.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:54 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:54.915+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:55.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:55.028+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:55.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:55.028+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:55.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:55.028+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:55.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:55.028+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T05:59:55.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.969+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:55.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.969+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:55.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:54 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:54.969+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:55.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:55.513+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:55.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:55.513+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:55.895 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:55.513+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:55.895 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:55 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:55.513+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:30.663667+0000 front 2024-09-19T05:59:30.663717+0000 (oldest deadline 2024-09-19T05:59:54.763782+0000) 2024-09-19T05:59:55.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:55 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:55.501+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:55.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:55 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:55.501+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:55.975 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:55.915+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:55.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:55.915+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:55.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:55.676+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:55.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:55.676+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:55.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:55.676+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:56.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:56.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:56.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:56.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:56.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:56.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:56.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:56.017+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T05:59:56.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.002+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:56.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.002+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:56.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.002+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:56.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:56.475+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.159360+0000 front 2024-09-19T05:59:08.159447+0000 (oldest deadline 2024-09-19T05:59:31.659568+0000) 2024-09-19T05:59:56.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:56.475+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.960737+0000 front 2024-09-19T05:59:13.960809+0000 (oldest deadline 2024-09-19T05:59:38.060567+0000) 2024-09-19T05:59:56.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:56.475+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:22.463399+0000 front 2024-09-19T05:59:22.463379+0000 (oldest deadline 2024-09-19T05:59:47.763008+0000) 2024-09-19T05:59:56.894 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:56 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6[56393]: 2024-09-19T05:59:56.475+0000 7ff74f9b9640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:30.663667+0000 front 2024-09-19T05:59:30.663717+0000 (oldest deadline 2024-09-19T05:59:54.763782+0000) 2024-09-19T05:59:56.945 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:56.640+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:56.945 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:56.640+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:56.945 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:56.640+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:56.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:56.453+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:56.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:56.453+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:57.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:56.944+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:57.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:56.944+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:57.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:56 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:56.944+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T05:59:57.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:57 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:57.011+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:57.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:57 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:57.011+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:57.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:57 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:57.011+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:57.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:57 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:57.011+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T05:59:57.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.957+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:57.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.957+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:57.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:56 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:56.957+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:57.863 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:57 smithi038 podman[98541]: 2024-09-19 05:59:57.569215596 +0000 UTC m=+5.577516053 container died e58537546f30703d22d08c2001fca8ec1a07e186db01fa75c66ce6d6f6a641d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T05:59:57.900 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:57.608+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:57.901 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:57.608+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:57.901 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:57.608+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:57.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:57.472+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:57.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:57.472+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:57.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:57.472+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98541]: 2024-09-19 05:59:58.011325422 +0000 UTC m=+6.019625879 container cleanup e58537546f30703d22d08c2001fca8ec1a07e186db01fa75c66ce6d6f6a641d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 bash[98541]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:58 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:58.056+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:58 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:58.056+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:58 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:58.056+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:58.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:58 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:58.056+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T05:59:58.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:57.899+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:58.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:57.899+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:58.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:57 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:57.899+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T05:59:58.436 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:57.970+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:58.436 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:57.970+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:58.436 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:57 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:57.970+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:58.642 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98557]: 2024-09-19 05:59:58.221375459 +0000 UTC m=+0.645283874 container remove e58537546f30703d22d08c2001fca8ec1a07e186db01fa75c66ce6d6f6a641d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T05:59:58.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:58.434+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:58.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:58.434+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:58.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:58.434+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T05:59:58.881 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:58.568+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:58.882 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:58.568+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:58.882 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:58.568+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:58.882 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:58.568+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T05:59:58.893 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98647]: 2024-09-19 05:59:58.543921942 +0000 UTC m=+0.016285499 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:59:58.893 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98647]: 2024-09-19 05:59:58.66569405 +0000 UTC m=+0.138057638 container create 8d9f5880290473f1acbe4a0d531bc8ace77c96666044162f3a79c2c328e81c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T05:59:59.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:58.880+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T05:59:59.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:58.880+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T05:59:59.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:58 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:58.880+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T05:59:59.389 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:58.972+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T05:59:59.389 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:58.973+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T05:59:59.389 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:58 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:58.973+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T05:59:59.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:59 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:59.087+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:59 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:59.087+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:59 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:59.087+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 05:59:59 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T05:59:59.087+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98647]: 2024-09-19 05:59:58.941860302 +0000 UTC m=+0.414223859 container init 8d9f5880290473f1acbe4a0d531bc8ace77c96666044162f3a79c2c328e81c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:58 smithi038 podman[98647]: 2024-09-19 05:59:58.945212439 +0000 UTC m=+0.417575990 container start 8d9f5880290473f1acbe4a0d531bc8ace77c96666044162f3a79c2c328e81c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:59:59.394 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:59 smithi038 podman[98647]: 2024-09-19 05:59:59.000151614 +0000 UTC m=+0.472515182 container attach 8d9f5880290473f1acbe4a0d531bc8ace77c96666044162f3a79c2c328e81c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T05:59:59.395 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 19 05:59:59 smithi038 podman[98647]: 2024-09-19 05:59:59.14107869 +0000 UTC m=+0.613442237 container died 8d9f5880290473f1acbe4a0d531bc8ace77c96666044162f3a79c2c328e81c1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T05:59:59.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:59.387+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T05:59:59.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:59.387+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T05:59:59.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T05:59:59.387+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T05:59:59.763 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T05:59:59.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:59:59.853 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T05:59:59.853 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T05:59:59.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.7 2024-09-19T05:59:59.864 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:59.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T05:59:59.864 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:59.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T05:59:59.864 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:59.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T05:59:59.864 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T05:59:59.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:00.162 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:59.862+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:00.162 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:59.862+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:00.162 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:59 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T05:59:59.862+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:00.162 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 05:59:59 smithi047 systemd[1]: Stopping Ceph osd.7 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:00.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:59.954+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:00.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:59.954+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:00.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 05:59:59 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T05:59:59.954+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:00.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:00 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:00.120+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:00.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:00 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:00.120+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:00.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:00 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:00.120+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:00.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:00 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:00.120+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:00.441 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.160+0000 7fc778ef9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:00.441 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.160+0000 7fc778ef9640 -1 osd.7 95 *** Got signal Terminated *** 2024-09-19T06:00:00.441 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.160+0000 7fc778ef9640 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:00:00.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:00.386+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:00.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:00.386+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:00.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:00.386+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:00.725 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:00.580+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:00.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:00.580+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:00.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:00.580+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:00.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:00.580+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:01.225 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.877+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:01.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.877+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:01.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:00.877+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:01.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:00.965+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:01.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:00.965+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:01.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:00 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:00.965+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:01.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:01 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:01.089+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:01.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:01 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:01.089+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:01.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:01 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:01.089+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:01.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:01 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:01.089+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:01.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:01 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:01.360+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:01.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:01 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:01.360+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:01.698 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:01 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:01.360+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:01.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:01.586+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:01.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:01.586+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:01.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:01.586+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:01.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:01.586+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:02.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:01.892+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:02.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:01.892+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:02.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:01 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:01.892+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:02.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:02.004+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:02.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:02.004+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:02.362 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:02.004+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:02.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:02 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:02.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:02.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:02 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:02.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:02.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:02 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:02.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:02.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:02 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:02.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:02.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:02.361+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:02.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:02.361+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:02.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:02 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:02.361+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:02.856 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:02.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:02.856 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:02.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:02.857 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:02.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:02.857 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:02.581+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:03.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:02.855+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:03.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:02.855+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:03.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:02 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:02.855+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:03.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:03.092+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:03.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:03.092+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:03.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:03.092+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:03.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:03 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:03.092+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:03.409 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:03.039+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:03.409 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:03.039+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:03.409 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:03.039+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:03.683 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:03.407+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:03.683 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:03.407+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:03.683 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:03 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:03.407+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:03.871 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:03.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:03.871 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:03.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:03.871 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:03.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:03.871 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:03.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:03.871 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:03.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:04.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:03.868+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:04.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:03.868+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:04.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:03.868+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:04.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:03 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:03.868+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.207018+0000 front 2024-09-19T05:59:38.207072+0000 (oldest deadline 2024-09-19T06:00:02.907179+0000) 2024-09-19T06:00:04.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:04.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:04.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:04.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:04.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:04.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:04.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:04 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:04.102+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:04.431 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:04.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:04.432 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:04.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:04.432 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:04.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:04.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:04.430+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:04.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:04.430+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:04.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:04 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:04.430+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:04.917 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:04.558+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:04.917 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:04.558+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:04.917 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:04.558+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:04.917 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:04.558+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:04.917 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:04.558+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:05.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:04.914+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.303583+0000 front 2024-09-19T05:59:17.303552+0000 (oldest deadline 2024-09-19T05:59:42.003733+0000) 2024-09-19T06:00:05.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:04.914+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.304734+0000 front 2024-09-19T05:59:24.304885+0000 (oldest deadline 2024-09-19T05:59:46.604711+0000) 2024-09-19T06:00:05.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:04.914+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:31.206010+0000 front 2024-09-19T05:59:31.206077+0000 (oldest deadline 2024-09-19T05:59:56.506154+0000) 2024-09-19T06:00:05.226 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7[53001]: 2024-09-19T06:00:04.914+0000 7fc77550f640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.207018+0000 front 2024-09-19T05:59:38.207072+0000 (oldest deadline 2024-09-19T06:00:02.907179+0000) 2024-09-19T06:00:05.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:05.098+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:05.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:05.098+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:05.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:05.098+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:05.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:05 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:05.098+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:05.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:05.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:05.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:05.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:05.450+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:05.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:05.450+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:05.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:05.450+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:05.948 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:05 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:05.450+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:05.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:05 smithi047 podman[70925]: 2024-09-19 06:00:05.686498776 +0000 UTC m=+5.641143863 container died 6d064b23ebbc8066d98476366672557f6bc81e6e6aa6e2880fb521b1e1406244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T06:00:05.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:05 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:05.560+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:05.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:05 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:05.560+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:05.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:05 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:05.560+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:05.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:05 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:05.560+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:05.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:05 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:05.560+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:06.329 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[70925]: 2024-09-19 06:00:06.093700173 +0000 UTC m=+6.048345261 container cleanup 6d064b23ebbc8066d98476366672557f6bc81e6e6aa6e2880fb521b1e1406244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T06:00:06.329 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 bash[70925]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7 2024-09-19T06:00:06.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:06.097+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:06.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:06.097+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:06.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:06.097+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:06.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:06.097+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:06.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:06 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:06.097+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:06.441 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:06.117+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:06.442 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:06.117+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:06.442 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:06.117+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:06.442 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:06.117+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:06.669 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[70954]: 2024-09-19 06:00:06.328264988 +0000 UTC m=+0.638693467 container remove 6d064b23ebbc8066d98476366672557f6bc81e6e6aa6e2880fb521b1e1406244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:00:06.670 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:06 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:06.552+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:06.670 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:06 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:06.552+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:06.670 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:06 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:06.552+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:06.670 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:06 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:06.552+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:06.670 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:06 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:06.552+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:06.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:06.439+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:06.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:06.439+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:06.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:06.439+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:06.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:06 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:06.439+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:06.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[71043]: 2024-09-19 06:00:06.571225573 +0000 UTC m=+0.015788956 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:06.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[71043]: 2024-09-19 06:00:06.670669338 +0000 UTC m=+0.115232707 container create 528ee2320889a95e7651055e9727b0b19dac914e0a19091d292b40da22bca909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T06:00:06.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[71043]: 2024-09-19 06:00:06.892090995 +0000 UTC m=+0.336654367 container init 528ee2320889a95e7651055e9727b0b19dac914e0a19091d292b40da22bca909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T06:00:06.976 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[71043]: 2024-09-19 06:00:06.895518801 +0000 UTC m=+0.340082170 container start 528ee2320889a95e7651055e9727b0b19dac914e0a19091d292b40da22bca909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:00:07.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:07.072+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:07.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:07.072+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:07.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:07.072+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:07.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:07.072+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:07.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:07 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:07.072+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:07.411 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:07.104+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:07.411 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:07.104+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:07.411 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:07.104+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:07.411 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:07.104+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:07.444 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:06 smithi047 podman[71043]: 2024-09-19 06:00:06.999211366 +0000 UTC m=+0.443774749 container attach 528ee2320889a95e7651055e9727b0b19dac914e0a19091d292b40da22bca909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:00:07.444 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 19 06:00:07 smithi047 podman[71043]: 2024-09-19 06:00:07.083172289 +0000 UTC m=+0.527735669 container died 528ee2320889a95e7651055e9727b0b19dac914e0a19091d292b40da22bca909 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T06:00:07.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:07.410+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:07.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:07.410+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:07.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:07.410+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:07.697 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:07 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:07.410+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:07.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:07 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:07.559+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:07.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:07 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:07.559+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:07.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:07 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:07.559+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:07.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:07 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:07.559+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:07.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:07 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:07.559+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:07.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T06:00:07.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:07.867 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T06:00:07.867 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T06:00:07.867 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.8 2024-09-19T06:00:08.187 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:07 smithi179 systemd[1]: Stopping Ceph osd.8 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:08.188 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:08.075+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:08.188 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:08.075+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:08.188 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:08.075+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:08.188 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:08.075+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:08.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:08.034+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:08.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:08.034+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:08.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:08.034+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:08.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:08.034+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:08.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:08 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:08.034+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:08.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.186+0000 7f1dd1557640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:08.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.186+0000 7f1dd1557640 -1 osd.8 95 *** Got signal Terminated *** 2024-09-19T06:00:08.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.186+0000 7f1dd1557640 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:00:08.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.458+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:08.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.458+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:08.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.458+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:08.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:08 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:08.458+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:08.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:08.517+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:08.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:08.517+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:08.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:08.517+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:08.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:08.517+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:08.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:08.517+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:09.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:09.025+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:09.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:09.025+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:09.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:09.025+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:09.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:09.025+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:09.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:09 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:09.025+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:09.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:09.089+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:09.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:09.089+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:09.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:09.089+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:09.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:09.089+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:09.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:09.502+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:09.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:09.502+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:09.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:09.502+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:09.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:09 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:09.502+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:09.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:09.499+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:09.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:09.499+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:09.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:09.499+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:09.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:09.499+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:09.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:09 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:09.499+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:10.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:10 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:10.067+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:10.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:10.052+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:10.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:10.052+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:10.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:10.052+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:10.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:10.052+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:10.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:10.052+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:10.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:10.474+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:10.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:10.474+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:10.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:10.474+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:10.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:10 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:10.474+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:10.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:10.536+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:10.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:10.536+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:10.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:10.536+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:10.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:10.536+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:10.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:10 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:10.536+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:11.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:11.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:11.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:11.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:11.321 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:11.322 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:11 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:11.063+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:11.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:11.058+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:11.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:11.058+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:11.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:11.058+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:11.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:11.058+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:11.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:11.058+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:11.948 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:11.463+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:11.949 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:11.463+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:11.949 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:11.463+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:11.949 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:11 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:11.463+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:11.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:11.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:11.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:11.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:11.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:11.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:11.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:11.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:11.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:11 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:11.549+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:12.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:12 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:12.042+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:12.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:12.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:12.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:12.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:12.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:12.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:12.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:12.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:12.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:12.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:12.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:12.499+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:12.366267+0000 front 2024-09-19T05:59:12.366155+0000 (oldest deadline 2024-09-19T05:59:39.966721+0000) 2024-09-19T06:00:12.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:12.499+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:24.067560+0000 front 2024-09-19T05:59:24.067518+0000 (oldest deadline 2024-09-19T05:59:49.367632+0000) 2024-09-19T06:00:12.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:12.499+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.268616+0000 front 2024-09-19T05:59:32.268718+0000 (oldest deadline 2024-09-19T05:59:56.869229+0000) 2024-09-19T06:00:12.948 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8[54082]: 2024-09-19T06:00:12.499+0000 7f1dc9b65640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:39.770643+0000 front 2024-09-19T05:59:39.770491+0000 (oldest deadline 2024-09-19T06:00:04.470362+0000) 2024-09-19T06:00:12.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:12.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:12.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:12.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:12.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:12.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:12 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:12.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:13.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:13.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:13.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:13.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:13.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:13.395 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:13 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:13.085+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:13.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:13 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:13.130+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:13.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:13 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:13.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:13.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:13 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:13.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:13.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:13 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:13.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:13.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:13 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:13.131+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:13.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:13 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:13.592+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:13.988 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:13 smithi179 podman[72840]: 2024-09-19 06:00:13.707498372 +0000 UTC m=+5.643976265 container died aa43eed2a247249ff2e68868e166f437fb9fa2a72c2eb89005f69d392d8bacdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T06:00:14.372 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72840]: 2024-09-19 06:00:14.07790482 +0000 UTC m=+6.014382725 container cleanup aa43eed2a247249ff2e68868e166f437fb9fa2a72c2eb89005f69d392d8bacdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:00:14.372 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 bash[72840]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8 2024-09-19T06:00:14.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:14 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:14.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:14.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:14 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:14.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:14.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:14 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:14.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:14.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:14 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:14.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:14.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:14 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:14.088+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:14.393 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:14.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:14.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:14.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:14.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:14.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:14 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:14.053+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:14.690 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72854]: 2024-09-19 06:00:14.371149808 +0000 UTC m=+0.652863185 container remove aa43eed2a247249ff2e68868e166f437fb9fa2a72c2eb89005f69d392d8bacdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:14.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72939]: 2024-09-19 06:00:14.595941786 +0000 UTC m=+0.015699653 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:14.947 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72939]: 2024-09-19 06:00:14.738873688 +0000 UTC m=+0.158631543 container create 22d9da984ccf7cb75d180a604c5b4421d6653f8504189e0ff46fd30e87933cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:00:14.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:14.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:14.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:14.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:14.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:14.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:14 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:14.617+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:15.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.013+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:15.439 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72939]: 2024-09-19 06:00:14.974677375 +0000 UTC m=+0.394435244 container init 22d9da984ccf7cb75d180a604c5b4421d6653f8504189e0ff46fd30e87933cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:14 smithi179 podman[72939]: 2024-09-19 06:00:14.97789263 +0000 UTC m=+0.397650486 container start 22d9da984ccf7cb75d180a604c5b4421d6653f8504189e0ff46fd30e87933cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:15 smithi179 podman[72939]: 2024-09-19 06:00:15.074661305 +0000 UTC m=+0.494419178 container attach 22d9da984ccf7cb75d180a604c5b4421d6653f8504189e0ff46fd30e87933cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.8.smithi179.stdout:Sep 19 06:00:15 smithi179 podman[72939]: 2024-09-19 06:00:15.168812832 +0000 UTC m=+0.588570689 container died 22d9da984ccf7cb75d180a604c5b4421d6653f8504189e0ff46fd30e87933cb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:15 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:15.046+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:15 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:15.046+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:15 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:15.046+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:15 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:15.046+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:15.499 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:15 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:15.046+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:15.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T06:00:15.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:15.873 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T06:00:15.873 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T06:00:15.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.9 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:15.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:15 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:15.605+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 systemd[1]: Stopping Ceph osd.9 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:16.184 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:15 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:15.970+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:16.387 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:16.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:16.387 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:16.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:16.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:16.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:16.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:16.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:16.388 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:16.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:16.644 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.182+0000 7f6354728640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:16.644 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.182+0000 7f6354728640 -1 osd.9 95 *** Got signal Terminated *** 2024-09-19T06:00:16.644 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.182+0000 7f6354728640 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:16.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:16 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:16.627+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:17.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:16 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:16.968+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:17.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:17 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:17.098+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:17.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:17 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:17.098+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:17.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:17 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:17.098+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:17.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:17 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:17.098+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:17.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:17 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:17.098+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:17.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:17.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:17.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:17.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:17.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:17.977 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:17 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:17.591+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:18.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:17 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:17.928+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:18.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:18 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:18.123+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:18.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:18 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:18.123+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:18.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:18 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:18.123+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:18.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:18 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:18.123+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:18.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:18 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:18.123+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:18.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:18 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:18.571+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:19.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:18 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:18.957+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:19.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:19.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:19.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:19.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:19.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:19.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:19 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:19.161+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:19.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:19 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:19.547+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:20.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:19 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:19.944+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:20.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:20.193+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:20.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:20 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:20.550+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.402026+0000 front 2024-09-19T05:59:08.402044+0000 (oldest deadline 2024-09-19T05:59:33.702148+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:13.703015+0000 front 2024-09-19T05:59:13.702737+0000 (oldest deadline 2024-09-19T05:59:37.802823+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:23.104096+0000 front 2024-09-19T05:59:23.104003+0000 (oldest deadline 2024-09-19T05:59:49.004173+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:33.005652+0000 front 2024-09-19T05:59:33.005695+0000 (oldest deadline 2024-09-19T05:59:54.105849+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.407769+0000 front 2024-09-19T05:59:40.407784+0000 (oldest deadline 2024-09-19T06:00:05.107681+0000) 2024-09-19T06:00:21.394 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:20 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9[63366]: 2024-09-19T06:00:20.895+0000 7f6350d3e640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:45.608673+0000 front 2024-09-19T05:59:45.608615+0000 (oldest deadline 2024-09-19T06:00:09.708384+0000) 2024-09-19T06:00:21.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:21.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:21.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:21.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:21.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:21.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:21 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:21.152+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:21.868 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:21 smithi038 podman[98763]: 2024-09-19 06:00:21.610004602 +0000 UTC m=+5.529475831 container died 2fef1d60679fe75c51f070ec3b434febab07de75ee851b446336d22d37d74a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:21.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:21 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:21.556+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:22.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:21 smithi038 podman[98763]: 2024-09-19 06:00:21.948800086 +0000 UTC m=+5.868271315 container cleanup 2fef1d60679fe75c51f070ec3b434febab07de75ee851b446336d22d37d74a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T06:00:22.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:21 smithi038 bash[98763]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9 2024-09-19T06:00:22.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98779]: 2024-09-19 06:00:22.108321238 +0000 UTC m=+0.487469688 container remove 2fef1d60679fe75c51f070ec3b434febab07de75ee851b446336d22d37d74a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T06:00:22.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:22.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:22.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:22.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:22.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:22.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:22 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:22.181+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:22.855 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.423558283 +0000 UTC m=+0.016753276 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:22.855 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.525886986 +0000 UTC m=+0.119081967 container create e67a96941c2585046867cb9c65689ad60ac711c3f7d227e93e5c7c33b3212432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T06:00:22.855 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.76212244 +0000 UTC m=+0.355317429 container init e67a96941c2585046867cb9c65689ad60ac711c3f7d227e93e5c7c33b3212432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T06:00:22.855 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.765668019 +0000 UTC m=+0.358863001 container start e67a96941c2585046867cb9c65689ad60ac711c3f7d227e93e5c7c33b3212432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:00:22.855 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.853911822 +0000 UTC m=+0.447106827 container attach e67a96941c2585046867cb9c65689ad60ac711c3f7d227e93e5c7c33b3212432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:00:22.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:22.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:22 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:22.524+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:23.144 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 19 06:00:22 smithi038 podman[98874]: 2024-09-19 06:00:22.966206314 +0000 UTC m=+0.559401314 container died e67a96941c2585046867cb9c65689ad60ac711c3f7d227e93e5c7c33b3212432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:23.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:23 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:23.166+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:23.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T06:00:23.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:23.593 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T06:00:23.593 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T06:00:23.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.10 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:23.873 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.554+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:23.874 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 systemd[1]: Stopping Ceph osd.10 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:23.874 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.871+0000 7fc0b0491640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:23.874 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.871+0000 7fc0b0491640 -1 osd.10 95 *** Got signal Terminated *** 2024-09-19T06:00:23.874 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:23 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:23.871+0000 7fc0b0491640 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:00:24.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:24.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:24.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:24.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:24.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:24.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:24.141+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:24.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:24.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:24 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:24.545+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:25.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:25.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:25.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:25.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:25.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:25.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:25 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:25.172+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:25.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:25 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:25.528+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:26.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:26.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:26.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:26.197 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:26.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:26.198 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:26 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:26.129+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:26.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:26 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:26.565+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:27.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:27.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:27.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:27.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:27.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:27.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:27 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:27.096+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:27.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:27 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:27.598+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:28.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:28.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:28.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:28.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:28.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:28.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:28 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:28.108+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:09.168642+0000 front 2024-09-19T05:59:09.168874+0000 (oldest deadline 2024-09-19T05:59:33.268599+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:17.071071+0000 front 2024-09-19T05:59:17.071103+0000 (oldest deadline 2024-09-19T05:59:41.771280+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:21.771782+0000 front 2024-09-19T05:59:21.771818+0000 (oldest deadline 2024-09-19T05:59:48.172245+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-19T05:59:32.873313+0000 front 2024-09-19T05:59:32.873101+0000 (oldest deadline 2024-09-19T05:59:58.172997+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:38.173479+0000 front 2024-09-19T05:59:38.173444+0000 (oldest deadline 2024-09-19T06:00:02.873490+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:46.374500+0000 front 2024-09-19T05:59:46.374536+0000 (oldest deadline 2024-09-19T06:00:12.274315+0000) 2024-09-19T06:00:28.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:28 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10[59544]: 2024-09-19T06:00:28.634+0000 7fc0acaa7640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:55.676106+0000 front 2024-09-19T05:59:55.676171+0000 (oldest deadline 2024-09-19T06:00:20.375905+0000) 2024-09-19T06:00:29.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:29.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:29.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:29.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:29.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:29.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:29 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:29.124+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:29.476 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:29 smithi047 podman[71147]: 2024-09-19 06:00:29.179659505 +0000 UTC m=+5.391060393 container died 80e0a31f40c580052e72ae7991fe996e9147f369f70f192b355fb8156dfa8dfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:00:29.975 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:29 smithi047 podman[71147]: 2024-09-19 06:00:29.47903005 +0000 UTC m=+5.690430926 container cleanup 80e0a31f40c580052e72ae7991fe996e9147f369f70f192b355fb8156dfa8dfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:00:29.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:29 smithi047 bash[71147]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10 2024-09-19T06:00:29.976 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:29 smithi047 podman[71163]: 2024-09-19 06:00:29.663453743 +0000 UTC m=+0.482311228 container remove 80e0a31f40c580052e72ae7991fe996e9147f369f70f192b355fb8156dfa8dfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T06:00:30.392 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:29.994590728 +0000 UTC m=+0.015516834 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:30.393 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:30.097690924 +0000 UTC m=+0.118617022 container create 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:30.393 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:30.316552419 +0000 UTC m=+0.337478524 container init 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:00:30.393 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:30.320143403 +0000 UTC m=+0.341069508 container start 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:00:30.393 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:30.391391832 +0000 UTC m=+0.412317940 container attach 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T06:00:30.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:30.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:30.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:30.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:30.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:30.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:30 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:30.140+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:30.726 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:30 smithi047 podman[71251]: 2024-09-19 06:00:30.513500288 +0000 UTC m=+0.534426387 container died 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T06:00:31.066 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 19 06:00:31 smithi047 podman[71274]: 2024-09-19 06:00:31.044544108 +0000 UTC m=+0.525666560 container remove 7770e404a8e14575f17e681a946f793a7c9ce6e8eaa5d827b2e7fd3cbeb7312e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T06:00:31.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T06:00:31.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:31.115 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T06:00:31.115 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T06:00:31.115 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.11 2024-09-19T06:00:31.424 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 systemd[1]: Stopping Ceph osd.11 for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.169+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.422+0000 7f388d5b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.422+0000 7f388d5b1640 -1 osd.11 95 *** Got signal Terminated *** 2024-09-19T06:00:31.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:31 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:31.422+0000 7f388d5b1640 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:00:32.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:32.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:32.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:32.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:32.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:32.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:32.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:32 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:32.208+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.232+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.232+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.232+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.232+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.232+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.233+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:33.599 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:33 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:33.233+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:34.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:34.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:34.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:34.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:34.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:34.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:34.448 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:34 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:34.191+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:35.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:35.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:35.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:35.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:35.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:35.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:35.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:35 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:35.241+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-19T05:59:08.122956+0000 front 2024-09-19T05:59:08.122939+0000 (oldest deadline 2024-09-19T05:59:32.822939+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-19T05:59:14.523985+0000 front 2024-09-19T05:59:14.523902+0000 (oldest deadline 2024-09-19T05:59:39.223908+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-09-19T05:59:25.025447+0000 front 2024-09-19T05:59:25.025430+0000 (oldest deadline 2024-09-19T05:59:49.125524+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-19T05:59:40.227850+0000 front 2024-09-19T05:59:40.227905+0000 (oldest deadline 2024-09-19T06:00:05.527757+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-09-19T05:59:47.828607+0000 front 2024-09-19T05:59:47.828666+0000 (oldest deadline 2024-09-19T06:00:09.528790+0000) 2024-09-19T06:00:36.697 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-19T05:59:56.430840+0000 front 2024-09-19T05:59:56.430769+0000 (oldest deadline 2024-09-19T06:00:18.730742+0000) 2024-09-19T06:00:36.698 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11[60776]: 2024-09-19T06:00:36.227+0000 7f3889bc7640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-19T06:00:05.132274+0000 front 2024-09-19T06:00:05.132318+0000 (oldest deadline 2024-09-19T06:00:30.432234+0000) 2024-09-19T06:00:37.183 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:36 smithi179 podman[73053]: 2024-09-19 06:00:36.899826811 +0000 UTC m=+5.591908412 container died 53c1b993e0e6ccefdc59827da35ac8c5cd65d80f28a639c2ae5fb6b4ed629112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:37.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:37 smithi179 podman[73053]: 2024-09-19 06:00:37.273079706 +0000 UTC m=+5.965161304 container cleanup 53c1b993e0e6ccefdc59827da35ac8c5cd65d80f28a639c2ae5fb6b4ed629112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:00:37.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:37 smithi179 bash[73053]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11 2024-09-19T06:00:37.885 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:37 smithi179 podman[73083]: 2024-09-19 06:00:37.532432681 +0000 UTC m=+0.631546740 container remove 53c1b993e0e6ccefdc59827da35ac8c5cd65d80f28a639c2ae5fb6b4ed629112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T06:00:38.178 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:37 smithi179 podman[73169]: 2024-09-19 06:00:37.787366141 +0000 UTC m=+0.015580762 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:00:38.178 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:37 smithi179 podman[73169]: 2024-09-19 06:00:37.891346499 +0000 UTC m=+0.119561112 container create c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:00:38.178 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:38 smithi179 podman[73169]: 2024-09-19 06:00:38.094317878 +0000 UTC m=+0.322532506 container init c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:38.178 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:38 smithi179 podman[73169]: 2024-09-19 06:00:38.097863819 +0000 UTC m=+0.326078444 container start c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T06:00:38.179 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:38 smithi179 podman[73169]: 2024-09-19 06:00:38.177367866 +0000 UTC m=+0.405582481 container attach c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T06:00:38.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:38 smithi179 podman[73169]: 2024-09-19 06:00:38.284201841 +0000 UTC m=+0.512416459 container died c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:00:38.836 INFO:journalctl@ceph.osd.11.smithi179.stdout:Sep 19 06:00:38 smithi179 podman[73192]: 2024-09-19 06:00:38.813949772 +0000 UTC m=+0.523539793 container remove c01bca3aba4d098eab6d57fd8668df5f93c42fa73526d6dc1cfae99719481663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T06:00:38.841 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T06:00:38.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:38.881 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T06:00:38.881 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T06:00:38.881 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.a 2024-09-19T06:00:39.203 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:38 smithi038 systemd[1]: Stopping Ceph mds.a for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:39.644 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: 2024-09-19T06:00:39.201+0000 7f89f1068640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:39.644 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:39 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a[70367]: 2024-09-19T06:00:39.201+0000 7f89f1068640 -1 mds.a *** got signal Terminated *** 2024-09-19T06:00:43.603 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:43 smithi038 podman[98988]: 2024-09-19 06:00:43.215157112 +0000 UTC m=+4.121376795 container died 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T06:00:43.894 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:43 smithi038 podman[98988]: 2024-09-19 06:00:43.718106141 +0000 UTC m=+4.624325820 container cleanup 6cce782df8a5b5835a67998034d915c05aebfdbf8a77c92fcb1bfa32a8ff2a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:00:43.894 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 19 06:00:43 smithi038 bash[98988]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-a 2024-09-19T06:00:44.099 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T06:00:44.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:44.139 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T06:00:44.139 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T06:00:44.139 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.d 2024-09-19T06:00:44.490 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:44 smithi038 systemd[1]: Stopping Ceph mds.d for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:44.894 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: 2024-09-19T06:00:44.488+0000 7f690090a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:44.894 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:44 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d[69182]: 2024-09-19T06:00:44.488+0000 7f690090a640 -1 mds.d *** got signal Terminated *** 2024-09-19T06:00:47.808 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:47 smithi038 podman[99118]: 2024-09-19 06:00:47.540044253 +0000 UTC m=+3.153635015 container died 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-19T06:00:48.074 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:47 smithi038 podman[99118]: 2024-09-19 06:00:47.88056525 +0000 UTC m=+3.494156042 container cleanup 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T06:00:48.075 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:47 smithi038 bash[99118]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d 2024-09-19T06:00:48.310 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T06:00:48.333 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:48 smithi038 podman[99131]: 2024-09-19 06:00:48.073490716 +0000 UTC m=+0.530686584 container remove 3241021ccf809acd643a3c76c413d5b22b21404038c73439ee342176cef7cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:48.334 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:48 smithi038 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-19T06:00:48.334 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:48 smithi038 systemd[1]: Stopped Ceph mds.d for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T06:00:48.334 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 19 06:00:48 smithi038 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.d.service: Consumed 2.700s CPU time. 2024-09-19T06:00:48.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:48.346 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T06:00:48.346 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T06:00:48.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.g 2024-09-19T06:00:48.643 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:48 smithi038 systemd[1]: Stopping Ceph mds.g for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:49.144 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: 2024-09-19T06:00:48.760+0000 7f479c66d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:49.144 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:48 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g[69985]: 2024-09-19T06:00:48.760+0000 7f479c66d640 -1 mds.g *** got signal Terminated *** 2024-09-19T06:00:51.874 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:51 smithi038 podman[99241]: 2024-09-19 06:00:51.535747416 +0000 UTC m=+2.966818414 container died 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T06:00:51.874 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:51 smithi038 podman[99241]: 2024-09-19 06:00:51.872981244 +0000 UTC m=+3.304052195 container cleanup 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-19T06:00:52.144 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:51 smithi038 bash[99241]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g 2024-09-19T06:00:52.144 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 19 06:00:52 smithi038 podman[99255]: 2024-09-19 06:00:52.069736754 +0000 UTC m=+0.530204048 container remove 6fb71d531d653c3ed1bebb12a316053a0d7157776bcdd33d5baa15cac3c0999d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T06:00:52.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T06:00:52.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:52.351 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T06:00:52.351 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T06:00:52.351 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.j 2024-09-19T06:00:52.644 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:52 smithi038 systemd[1]: Stopping Ceph mds.j for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:53.144 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j[68783]: 2024-09-19T06:00:52.680+0000 7fb02d876640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:53.144 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:52 smithi038 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j[68783]: 2024-09-19T06:00:52.680+0000 7fb02d876640 -1 mds.j *** got signal Terminated *** 2024-09-19T06:00:54.802 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:54 smithi038 podman[99361]: 2024-09-19 06:00:54.536527069 +0000 UTC m=+1.958032773 container died 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:00:55.144 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:54 smithi038 podman[99361]: 2024-09-19 06:00:54.889278528 +0000 UTC m=+2.310784230 container cleanup 558f3161bf1db43b6f7e1e3ee6d123b7e64adf4cf6a0da81995f3c37b3714377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:00:55.144 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 19 06:00:54 smithi038 bash[99361]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-j 2024-09-19T06:00:55.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T06:00:55.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:00:55.347 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T06:00:55.347 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T06:00:55.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.b 2024-09-19T06:00:55.656 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:55 smithi047 systemd[1]: Stopping Ceph mds.b for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:00:55.975 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b[65195]: 2024-09-19T06:00:55.653+0000 7fb86f505640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:00:55.976 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:55 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b[65195]: 2024-09-19T06:00:55.653+0000 7fb86f505640 -1 mds.b *** got signal Terminated *** 2024-09-19T06:00:59.805 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:59 smithi047 podman[71365]: 2024-09-19 06:00:59.532999135 +0000 UTC m=+3.980178883 container died 1451b31d87e9e072e6a1f407a9e80becc3a79fba32acc5475f69cd48d48603f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:01:00.164 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:59 smithi047 podman[71365]: 2024-09-19 06:00:59.887002637 +0000 UTC m=+4.334182383 container cleanup 1451b31d87e9e072e6a1f407a9e80becc3a79fba32acc5475f69cd48d48603f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:01:00.164 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 19 06:00:59 smithi047 bash[71365]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-b 2024-09-19T06:01:00.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T06:01:00.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:00.368 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T06:01:00.369 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T06:01:00.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.e 2024-09-19T06:01:00.725 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:00 smithi047 systemd[1]: Stopping Ceph mds.e for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:01.225 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: 2024-09-19T06:01:00.723+0000 7fc6f38d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:01.225 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:00 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e[64578]: 2024-09-19T06:01:00.723+0000 7fc6f38d2640 -1 mds.e *** got signal Terminated *** 2024-09-19T06:01:03.794 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:03 smithi047 podman[71483]: 2024-09-19 06:01:03.525059785 +0000 UTC m=+2.939983909 container died 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T06:01:04.225 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:03 smithi047 podman[71483]: 2024-09-19 06:01:03.87396177 +0000 UTC m=+3.288885894 container cleanup 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T06:01:04.226 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:03 smithi047 bash[71483]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e 2024-09-19T06:01:04.226 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 19 06:01:04 smithi047 podman[71522]: 2024-09-19 06:01:04.041773521 +0000 UTC m=+0.510158776 container remove 5a2449bb962015249f4c114dc4fa21e834d8f7ef5bc4d4203a937363e8837d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T06:01:04.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T06:01:04.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:04.311 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T06:01:04.311 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T06:01:04.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.h 2024-09-19T06:01:04.629 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:04 smithi047 systemd[1]: Stopping Ceph mds.h for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:04.975 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: 2024-09-19T06:01:04.627+0000 7f6f94872640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:04.975 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:04 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h[64883]: 2024-09-19T06:01:04.627+0000 7f6f94872640 -1 mds.h *** got signal Terminated *** 2024-09-19T06:01:07.832 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:07 smithi047 podman[71633]: 2024-09-19 06:01:07.52862648 +0000 UTC m=+2.993977063 container stop 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:01:07.832 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:07 smithi047 podman[71633]: 2024-09-19 06:01:07.554344688 +0000 UTC m=+3.019695268 container died 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:01:08.225 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:07 smithi047 podman[71633]: 2024-09-19 06:01:07.887881793 +0000 UTC m=+3.353232366 container cleanup 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:01:08.226 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:07 smithi047 bash[71633]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h 2024-09-19T06:01:08.226 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 19 06:01:08 smithi047 podman[71644]: 2024-09-19 06:01:08.039130051 +0000 UTC m=+0.502977802 container remove 3b394193efccfcb4bf450c9672cca574837a76198106f8d946c63de0c001e646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:01:08.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T06:01:08.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:08.305 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T06:01:08.306 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T06:01:08.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.k 2024-09-19T06:01:08.601 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:08 smithi047 systemd[1]: Stopping Ceph mds.k for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:08.976 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k[64262]: 2024-09-19T06:01:08.599+0000 7fcc0a314640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:08.976 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:08 smithi047 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k[64262]: 2024-09-19T06:01:08.599+0000 7fcc0a314640 -1 mds.k *** got signal Terminated *** 2024-09-19T06:01:11.760 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:11 smithi047 podman[71754]: 2024-09-19 06:01:11.501640661 +0000 UTC m=+2.976172670 container died faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T06:01:12.011 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:11 smithi047 podman[71754]: 2024-09-19 06:01:11.832770534 +0000 UTC m=+3.307302530 container cleanup faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:01:12.011 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:11 smithi047 bash[71754]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k 2024-09-19T06:01:12.011 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:12 smithi047 podman[71767]: 2024-09-19 06:01:12.008667344 +0000 UTC m=+0.504614556 container remove faf15a7c0b19897122f97e86637655c4424049ab7d3437592522091bd680b38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-k, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T06:01:12.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T06:01:12.264 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:12 smithi047 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-19T06:01:12.264 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:12 smithi047 systemd[1]: Stopped Ceph mds.k for 44ff0e58-7648-11ef-bceb-c7b262605968. 2024-09-19T06:01:12.264 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 19 06:01:12 smithi047 systemd[1]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.k.service: Consumed 1.164s CPU time. 2024-09-19T06:01:12.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:12.276 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T06:01:12.276 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T06:01:12.276 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.c 2024-09-19T06:01:12.565 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:12 smithi179 systemd[1]: Stopping Ceph mds.c for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:12.947 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c[66008]: 2024-09-19T06:01:12.562+0000 7f1365281640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:12.947 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:12 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c[66008]: 2024-09-19T06:01:12.562+0000 7f1365281640 -1 mds.c *** got signal Terminated *** 2024-09-19T06:01:15.882 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:15 smithi179 podman[73286]: 2024-09-19 06:01:15.525399498 +0000 UTC m=+3.061955398 container died f9d8e2f9dbede738e4d07d61234c7b07989c4bc58a670b4ad768db6bd5e8627a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T06:01:16.197 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:15 smithi179 podman[73286]: 2024-09-19 06:01:15.980511412 +0000 UTC m=+3.517067317 container cleanup f9d8e2f9dbede738e4d07d61234c7b07989c4bc58a670b4ad768db6bd5e8627a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T06:01:16.197 INFO:journalctl@ceph.mds.c.smithi179.stdout:Sep 19 06:01:15 smithi179 bash[73286]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-c 2024-09-19T06:01:16.419 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T06:01:16.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:16.455 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T06:01:16.455 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T06:01:16.455 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.f 2024-09-19T06:01:16.802 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:16 smithi179 systemd[1]: Stopping Ceph mds.f for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:17.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: 2024-09-19T06:01:16.800+0000 7f5f1eee2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:17.197 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:16 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f[65708]: 2024-09-19T06:01:16.800+0000 7f5f1eee2640 -1 mds.f *** got signal Terminated *** 2024-09-19T06:01:19.819 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:19 smithi179 podman[73420]: 2024-09-19 06:01:19.524940315 +0000 UTC m=+2.843934498 container died a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:01:20.136 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:19 smithi179 podman[73420]: 2024-09-19 06:01:19.909068144 +0000 UTC m=+3.228062324 container cleanup a1e8f5fb61cf7396075a2d144340b8d6be5de69fb92da2fcbafd07c7696fedd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T06:01:20.136 INFO:journalctl@ceph.mds.f.smithi179.stdout:Sep 19 06:01:19 smithi179 bash[73420]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-f 2024-09-19T06:01:20.296 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T06:01:20.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:20.329 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T06:01:20.329 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T06:01:20.329 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.i 2024-09-19T06:01:20.697 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:20 smithi179 systemd[1]: Stopping Ceph mds.i for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:20.697 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: 2024-09-19T06:01:20.637+0000 7f7adf067640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:20.697 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:20 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i[66312]: 2024-09-19T06:01:20.637+0000 7f7adf067640 -1 mds.i *** got signal Terminated *** 2024-09-19T06:01:23.797 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:23 smithi179 podman[73541]: 2024-09-19 06:01:23.535864463 +0000 UTC m=+2.994237039 container died 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T06:01:24.123 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:23 smithi179 podman[73541]: 2024-09-19 06:01:23.929660442 +0000 UTC m=+3.388033017 container cleanup 41d210b9626bffaaaa9d98d447391141d083c10e40850fd8a9a7babc77a174fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:01:24.123 INFO:journalctl@ceph.mds.i.smithi179.stdout:Sep 19 06:01:23 smithi179 bash[73541]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-i 2024-09-19T06:01:24.250 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T06:01:24.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:24.284 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T06:01:24.284 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T06:01:24.284 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.l 2024-09-19T06:01:24.447 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:24 smithi179 systemd[1]: Stopping Ceph mds.l for 44ff0e58-7648-11ef-bceb-c7b262605968... 2024-09-19T06:01:24.947 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l[65406]: 2024-09-19T06:01:24.591+0000 7f40a1ee4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:01:24.947 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:24 smithi179 ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l[65406]: 2024-09-19T06:01:24.591+0000 7f40a1ee4640 -1 mds.l *** got signal Terminated *** 2024-09-19T06:01:27.835 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:27 smithi179 podman[73661]: 2024-09-19 06:01:27.54108868 +0000 UTC m=+3.041837710 container died 4e7b10391970d1d3f69028d7df2a23ac191bfa21715805741b80e3d79bda3dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T06:01:28.197 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:27 smithi179 podman[73661]: 2024-09-19 06:01:27.908109166 +0000 UTC m=+3.408858197 container cleanup 4e7b10391970d1d3f69028d7df2a23ac191bfa21715805741b80e3d79bda3dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T06:01:28.197 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:27 smithi179 bash[73661]: ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l 2024-09-19T06:01:28.197 INFO:journalctl@ceph.mds.l.smithi179.stdout:Sep 19 06:01:28 smithi179 podman[73673]: 2024-09-19 06:01:28.075761197 +0000 UTC m=+0.533721860 container remove 4e7b10391970d1d3f69028d7df2a23ac191bfa21715805741b80e3d79bda3dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-44ff0e58-7648-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:01:28.282 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-44ff0e58-7648-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T06:01:28.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:01:28.314 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T06:01:28.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:01:28.526 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:31.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:01:31.289 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:33.756 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:01:33.956 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:36.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:01:36.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:01:36.452 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:01:36.478 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T06:01:36.480 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi038/crash 2024-09-19T06:01:36.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:01:36.517 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:01:36.517 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:01:36.519 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi047/crash 2024-09-19T06:01:36.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:01:36.558 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:01:36.558 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:01:36.560 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi179/crash 2024-09-19T06:01:36.561 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:01:36.614 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:01:36.615 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:01:36.615 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T06:01:36.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-19T06:01:36.656 INFO:tasks.cephadm:Compressing logs... 2024-09-19T06:01:36.656 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:01:36.699 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:01:36.702 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:01:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T06:01:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-19T06:01:36.724 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T06:01:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.9.log/var/log/ceph/ceph-client.admin.log: 2024-09-19T06:01:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:01:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:01:36.725 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T06:01:36.729 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr: 83.3% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log.gz/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.11.log: 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-19T06:01:36.730 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:01:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T06:01:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T06:01:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.a.log: 83.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:01:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T06:01:36.731 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:01:36.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T06:01:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T06:01:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T06:01:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.2.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.5.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr: 88.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T06:01:36.735 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T06:01:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.4.log: /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:01:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:01:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T06:01:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:01:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:01:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.y.log: 94.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T06:01:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T06:01:36.740 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:01:36.740 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T06:01:36.742 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:01:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:01:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T06:01:36.742 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T06:01:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log: 88.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:01:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T06:01:36.744 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.d.log: 87.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:01:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:01:36.746 INFO:teuthology.orchestra.run.smithi047.stderr: 93.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T06:01:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T06:01:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:01:36.748 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.z.log: 93.3%gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T06:01:36.748 INFO:teuthology.orchestra.run.smithi179.stderr: -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T06:01:36.755 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T06:01:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:01:36.766 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T06:01:36.767 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:01:36.781 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T06:01:36.782 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:01:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:01:36.808 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T06:01:36.818 INFO:teuthology.orchestra.run.smithi047.stderr: 93.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T06:01:36.821 INFO:teuthology.orchestra.run.smithi038.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:01:36.825 INFO:teuthology.orchestra.run.smithi038.stderr: 94.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T06:01:36.826 INFO:teuthology.orchestra.run.smithi047.stderr: 92.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T06:01:36.829 INFO:teuthology.orchestra.run.smithi038.stderr: 94.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T06:01:36.839 INFO:teuthology.orchestra.run.smithi047.stderr: 94.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T06:01:36.903 INFO:teuthology.orchestra.run.smithi179.stderr: 92.8% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T06:01:36.910 INFO:teuthology.orchestra.run.smithi179.stderr: 92.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T06:01:37.020 INFO:teuthology.orchestra.run.smithi179.stderr: 94.2% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T06:01:37.110 INFO:teuthology.orchestra.run.smithi038.stderr: 90.4% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T06:01:37.186 INFO:teuthology.orchestra.run.smithi179.stderr: 91.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T06:01:37.280 INFO:teuthology.orchestra.run.smithi047.stderr: 91.9% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T06:01:37.601 INFO:teuthology.orchestra.run.smithi038.stderr: 92.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T06:01:37.841 INFO:teuthology.orchestra.run.smithi047.stderr: 92.4% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T06:01:37.856 INFO:teuthology.orchestra.run.smithi179.stderr: 92.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T06:01:37.868 INFO:teuthology.orchestra.run.smithi038.stderr: 90.3% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T06:01:37.967 INFO:teuthology.orchestra.run.smithi038.stderr: 92.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T06:01:38.117 INFO:teuthology.orchestra.run.smithi047.stderr: 92.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T06:01:38.136 INFO:teuthology.orchestra.run.smithi179.stderr: 93.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T06:01:38.205 INFO:teuthology.orchestra.run.smithi038.stderr: 93.1% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T06:01:38.257 INFO:teuthology.orchestra.run.smithi047.stderr: 91.7% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T06:01:38.463 INFO:teuthology.orchestra.run.smithi038.stderr: 93.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T06:01:38.465 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-19T06:01:38.465 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m1.752s 2024-09-19T06:01:38.465 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m6.817s 2024-09-19T06:01:38.465 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.225s 2024-09-19T06:01:38.563 INFO:teuthology.orchestra.run.smithi047.stderr: 91.8% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T06:01:38.566 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-19T06:01:38.566 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m1.849s 2024-09-19T06:01:38.566 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m6.474s 2024-09-19T06:01:38.566 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.241s 2024-09-19T06:01:38.751 INFO:teuthology.orchestra.run.smithi179.stderr: 93.0% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T06:01:38.836 INFO:teuthology.orchestra.run.smithi179.stderr: 91.8% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T06:01:39.108 INFO:teuthology.orchestra.run.smithi179.stderr: 91.5% -- replaced with /var/log/ceph/44ff0e58-7648-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T06:01:39.110 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-09-19T06:01:39.110 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m2.394s 2024-09-19T06:01:39.110 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m9.654s 2024-09-19T06:01:39.110 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m0.327s 2024-09-19T06:01:39.111 INFO:tasks.cephadm:Archiving logs... 2024-09-19T06:01:39.111 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi038/log 2024-09-19T06:01:39.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T06:01:40.049 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi047/log 2024-09-19T06:01:40.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T06:01:41.053 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi179/log 2024-09-19T06:01:41.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T06:01:42.589 INFO:tasks.cephadm:Removing cluster... 2024-09-19T06:01:42.589 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force 2024-09-19T06:01:42.803 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:43.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force 2024-09-19T06:01:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:43.535 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 44ff0e58-7648-11ef-bceb-c7b262605968 --force 2024-09-19T06:01:43.730 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 44ff0e58-7648-11ef-bceb-c7b262605968 2024-09-19T06:01:44.001 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T06:01:44.001 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T06:01:44.021 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T06:01:44.037 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T06:01:44.052 INFO:tasks.cephadm:Teardown complete 2024-09-19T06:01:44.052 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T06:01:44.067 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T06:01:44.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T06:01:44.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T06:01:44.080 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T06:01:44.190 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T06:01:44.200 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T06:01:44.200 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:01:44.203 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:01:44.205 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:01:44.217 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:01:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:01:44.219 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 -141us[ -142us] +/- 37ms 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +451us[ +450us] +/- 52ms 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -332us[ -332us] +/- 76ms 2024-09-19T06:01:44.222 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 -141us[ -147us] +/- 37ms 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +437us[ +431us] +/- 52ms 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -340us[ -340us] +/- 76ms 2024-09-19T06:01:44.224 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:01:44.225 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:01:44.225 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-09-19T06:01:44.225 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 -139us[ -148us] +/- 37ms 2024-09-19T06:01:44.225 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +435us[ +426us] +/- 52ms 2024-09-19T06:01:44.226 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -336us[ -345us] +/- 76ms 2024-09-19T06:01:44.226 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:01:44.226 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T06:01:44.236 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T06:01:44.236 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T06:01:44.275 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T06:01:44.363 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T06:01:44.434 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T06:01:44.501 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:01:44.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:44.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:01:44.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:44.562 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:01:44.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:44.592 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T06:01:44.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T06:01:44.623 INFO:teuthology.task.internal:Duration was 2236.840333 seconds 2024-09-19T06:01:44.623 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T06:01:44.665 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T06:01:44.666 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T06:01:44.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T06:01:44.671 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T06:01:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:01:44.709 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:01:44.711 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:01:45.014 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T06:01:45.014 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-09-19T06:01:45.015 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T06:01:45.038 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-19T06:01:45.038 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T06:01:45.079 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-09-19T06:01:45.079 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T06:01:45.108 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T06:01:45.108 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T06:01:45.111 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T06:01:45.122 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T06:01:45.175 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T06:01:45.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T06:01:45.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T06:01:45.189 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T06:01:46.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T06:01:46.039 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T06:01:46.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T06:01:46.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T06:01:46.095 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T06:01:46.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T06:01:46.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.156 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-09-19T06:01:46.163 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-19T06:01:46.186 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-09-19T06:01:46.210 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:46.225 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:46.243 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T06:01:46.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:01:46.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T06:01:46.276 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T06:01:46.277 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi038 2024-09-19T06:01:46.277 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T06:01:46.333 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi047 2024-09-19T06:01:46.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T06:01:46.374 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911677/remote/smithi179 2024-09-19T06:01:46.374 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T06:01:46.411 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T06:01:46.411 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T06:01:46.414 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T06:01:46.417 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T06:01:46.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T06:01:46.475 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T06:01:46.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T06:01:46.512 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T06:01:46.512 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T06:01:46.514 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T06:01:46.517 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T06:01:46.527 INFO:teuthology.orchestra.run.smithi038.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 06:01 /home/ubuntu/cephtest 2024-09-19T06:01:46.532 INFO:teuthology.orchestra.run.smithi179.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 06:01 /home/ubuntu/cephtest 2024-09-19T06:01:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 06:01 /home/ubuntu/cephtest 2024-09-19T06:01:46.533 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T06:01:46.557 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T06:01:46.644 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2236.8403334617615 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T06:01:46.644 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T06:01:46.724 INFO:teuthology.run:pass